Human factors evaluation in maritime virtual simulators using mobile EEG-based neuroimaging

Neuro-ergonomics using mobile electroencephalogram (EEG)-based neuroimaging is a new area of Brain-Computer Interaction (BCI) applications. We propose and develop an EEG-based system to monitor and analyze human factors measurements in maritime simulators. The EEG is used as a tool to monitor and record the brain states of subjects during human factors study experiments. In traditional human factors studies, the data of mental workload, stress, and emotion are obtained through questionnaires that are administered upon completion of some task/tasks or the whole experiment. However, this method only offers the evaluation of overall feelings of subjects during the task performance in the simulators. Real-time EEG-based human factors evaluation in maritime virtual simulator allows researchers to analyze the changes of subjects' brain states during the performance of various navigational tasks under different environmental and collaborative scenarios. Machine learning techniques are applied to the EEG data to recognize levels of mental workload, stress and emotions. By utilizing the proposed EEG-based system, true understanding of subjects working pattern can be obtained. Based on the analyses of the objective real-time data together with the subjective feedback from the subjects, we are able to reliably evaluate human factors during experiments in simulator. We describe real-time algorithms of emotion recognition, mental workload, and stress recognition from EEG and its integration in the cadets/captains stress assessment systems. We design a simulator-based experiment to record EEG signals of cadets, from which we recognize the changes of their emotions, mental workload, and stress levels during the task performance. We recorded EEG of 12 participants using Emotiv device in maritime simulator. The participants went through four exercises (around 30 minutes per exercise) with 20-minutes break in between. The exercises were with increasing difficulty levels and shuffled to be given to the participants. Videos were taken to analyze the behavioral data of the participants and used to label EEG data. Emotion, workload and stress levels are calculated from EEG recording with the time resolution 1 sec. From the preliminary case study it can be seen that there is a correlation between the EEG-based emotion recognition results (in terms of timing and magnitude) and the events that were happening in the simulator.