iSAM - towards a cost-efficient and unobtrusive experimental setup for situational awareness measurement in administrative crisis management exercises

Crisis management exercises provide a unique opportunity to study situational awareness in real-time scenarios. However, the federated structure of large-scale crisis management, involving multiple decentralized teams, imposes additional challenges for researchers aiming to capture comprehensive situational awareness data. Moreover, since the primary goal of these exercises is operational readiness rather than academic research, any experimental setup must be non-intrusive and place minimal extra-demand on participants. In a future exercise, we will be allowed to measure the situational awareness of crisis management staff members to evaluate a newly developed crisis management dashboard. To do so, we need to take both theoretical, as well as practical constraints into account, as we may participate in the exercise, yet must not disturb the exercise itself. To this date, neither methodology nor software tools are available to measure the situational awareness of spatially distributed crisis managers. In this paper we introduce such methodology, which requires a balancing act between scientific and practical requirements, where some scientific standards are contradicting practical circumstances and a reasonable experimental setup must be created. To achieve this, we present a methodology supported by software that allows situational awareness assessments. Given that crisis managers operate spatially distributed during a large-scale disaster or exercise and that we lack the resources to be physically present at all locations, our introduced software aims to conduct the experiment without a researcher in every situation room. This approach facilitates comprehensive data collection and analysis without setting up a complex research environment in every situation room, thereby allowing us to meet both scientific and operational objectives effectively.