LUMOS: Self-supervised Pretraining for Multimodal Task Load Estimation in Adaptive Instructional Systems

Accurate task load (TL) detection is essential for adaptive instructional systems (AIS) that tailor instruction to learner states, yet progress is hindered by costly data collection and scarce, noisy annotations. Recordings often contain substantial unlabeled segments, like rest periods, training phases, questionnaire completion, that supervised methods cannot exploit. We present LUMOS (Learning from Unlabeled Multimodal Observations in Task Load Detection Systems), a self-supervised learning framework that leverages these unlabeled multimodal physiological and behavioral signals to learn transferable representations for TL estimation. LUMOS employs discriminative self-supervised objectives, including MoCo, SimCLR, SwAV, and Barlow Twins, with modality-aware augmentations across electrocardiography, photoplethysmography, electrodermal activity, respiration, electromyography, skin temperature, pupil diameter, and eye movement data. We evaluate LUMOS on 96 sessions spanning n-back tests, driving simulation, and gaming tasks (Overcooked! 2, Hogwarts Legacy) using ConvNeXt, xLSTM, and Transformer backbones. MoCo pretrained models achieve the highest accuracy (0.79 ± 0.01), outperforming fully supervised baselines (0.72 ± 0.03) and demonstrating robust generalization across domains. Gains are most pronounced in low label regimes, where self-supervised initialization consistently outperforms training from scratch. Our results demonstrate that contrastive pretraining on unlabeled segments reduces annotation requirements and improves generalization, providing a practical foundation for TL-aware AIS applications.