Options
2019
Conference Paper
Titel
Evaluation of Diagnostic Rules for Real-Time Assessment of Mental Workload within a Dynamic Adaptation Framework
Abstract
Adaptive Instructional Systems (AIS) aim to support the learner by dynamically providing feedback tailored to the individual learner's needs. To select the appropriate type and level of support, the adaptive system gathers on-task information on learner performance and the learner's current mental state. The Real-Time Assessment of Multidimensional User State (RASMUS) is a rule-based diagnostic framework providing information on task performance and up to six user states affecting performance. The aim of this paper is to evaluate and optimize the existing diagnostic rules of RASMUS exemplary for the state of high mental workload. Modified rules were defined by performing receiver operator characteristic (ROC) curve analyses using the empirical data obtained in a prior validation experiment (N = 12). Subjective workload ratings served as comparative measure. Specifically, the analysis focused on optimizing the threshold values used to discriminate between critical and noncritical workload states for three physiological mental workload indicators, namely pupil dilation, heart rate variability and respiration rate. Subsequently, the prior validation study was repeated with N = 15 participants to evaluate the diagnostic accuracy of the initial and modified rule sets. Similar outcomes were found for the initial rule set confirming the temporal stability of the RASMUS diagnosis. However, contrary to expectations, the modified rule set showed less diagnostic accuracy when applied to the new data set. This result questions the practicality of ROC curve analysis for defining and optimizing rules in the context of physiological user state diagnosis.