Examining Joy of Use and Usability During Mobile Phone Interactions within a Multimodal Methods Approach

Objective: We investigate experienced joy of use (JoU) and usability using a multimodal methods approach by systematically varying mobile phone interactions. Methods: We combined subjective and objective measures to investigate whether positive emotional experiences and moments of joy during the interaction can be distinguished from neutral and negative emotional experiences. In a study with 30 participants, electroencephalography (EEG), electrocardiography (ECG), electrodermal activity (EDA), facial emotion recognition, and questionnaires were used. Results: There were greater positive experiences in interactions designed to elicit JoU, even under bad usability. We did not observe a difference between the conditions in the EEG indices. However, a higher heart rate and components in the EDA phasic response as well as facial muscle activity associated with anger were linked to good usability combined with no JoU. Conclusion: The multimodal methods approach reveals great potential to investigate JoU and usability in naturalistic scenarios. Application: The developed framework provides a groundwork to evaluate and improve interactions with technology. Thereby, users and their emotional experiences are placed at the centre when designing user interfaces. By detecting moments of joy, this approach can support a better understanding of how technology can be purposefully designed for joyful experiences.