Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Evaluation of Diagnostic Rules for Real-Time Assessment of Mental Workload within a Dynamic Adaptation Framework

: Bruder, A.; Schwarz, J.


Sottilare, Robert A.; Schwarz, Jessica:
Adaptive instructional systems. First International Conference, AIS 2019. Proceedings : Held as part of the 21st HCI International Conference, HCII 2019, Orlando, FL, USA, July 26-31, 2019
Cham: Springer Nature, 2019 (Lecture Notes in Computer Science 11597)
ISBN: 978-3-030-22340-3 (Print)
ISBN: 978-3-030-22341-0 (Online)
DOI: 10.1007/978-3-030-22341-0
International Conference on Adaptive Instructional Systems (AIS) <1, 2019, Orlando/Fla.>
International Conference on Human-Computer Interaction (HCI International) <21, 2019, Orlando/Fla.>
Conference Paper
Fraunhofer FKIE ()

Adaptive Instructional Systems (AIS) aim to support the learner by dynamically providing feedback tailored to the individual learner’s needs. To select the appropriate type and level of support, the adaptive system gathers on-task information on learner performance and the learner’s current mental state. The Real-Time Assessment of Multidimensional User State (RASMUS) is a rule-based diagnostic framework providing information on task performance and up to six user states affecting performance. The aim of this paper is to evaluate and optimize the existing diagnostic rules of RASMUS exemplary for the state of high mental workload. Modified rules were defined by performing receiver operator characteristic (ROC) curve analyses using the empirical data obtained in a prior validation experiment (N = 12). Subjective workload ratings served as comparative measure. Specifically, the analysis focused on optimizing the threshold values used to discriminate between critical and noncritical workload states for three physiological mental workload indicators, namely pupil dilation, heart rate variability and respiration rate. Subsequently, the prior validation study was repeated with N = 15 participants to evaluate the diagnostic accuracy of the initial and modified rule sets. Similar outcomes were found for the initial rule set confirming the temporal stability of the RASMUS diagnosis. However, contrary to expectations, the modified rule set showed less diagnostic accuracy when applied to the new data set. This result questions the practicality of ROC curve analysis for defining and optimizing rules in the context of physiological user state diagnosis.