Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Investigating static and sequential models for intervention-free selection using multimodal data of EEG and eye tracking

: Salous, M.; Putze, F.; Schultz, T.; Hild, Jutta; Beyerer, Jürgen


Association for Computing Machinery -ACM-; Association for Computing Machinery -ACM-, Special Interest Group on Computer and Human Interaction -SIGCHI-:
ICMI 2018, 20th ACM International Conference on Multimodal Interaction. Proceedings : MCPMD 2018, Workshop on Modeling Cognitive Processes from Multimodal Data, Boulder, Colorado, October 16th-20th, 2018
New York: ACM, 2018
ISBN: 978-1-4503-6072-2
ISBN: 978-1-4503-5692-3
Art. 7, 6 pp.
International Conference on Multimodal Interaction (ICMI) <20, 2018, Boulder/Colo.>
Workshop on Modeling Cognitive Processes from Multimodal Data (MCPMD) <2018, Boulder/Colo.>
Conference Paper
Fraunhofer IOSB ()
multimodal data; EEG; eye tracking; competition model; recall; precision

Multimodal data is increasingly used in cognitive prediction models to better analyze and predict different user cognitive processes. Classifiers based on such data, however, have different performance characteristics. We discuss in this paper an intervention-free selection task using multimodal data of EEG and eye tracking in three different models. We show that a sequential model, LSTM, is more sensitive but less precise than a static model SVM. Moreover, we introduce a confidence-based Competition-Fusion model using both SVM and LSTM. The fusion model further improves the recall compared to either SVM or LSTM alone, without decreasing precision compared to LSTM. According to the results, we recommend SVM for interactive applications which require minimal false positives (high precision), and recommend LSTM and highly recommend Competition-Fusion Model for applications which handle intervention-free selection requests in an additional post-processing step, requiring higher recall than precision.