Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Success is not Final; Failure is not Fatal - Task Success and User Experience in Interactions with Alexa, Google Assistant and Siri

: Kurz, Miriam; Brüggemeier, Birgit; Breiter, Michael

Postprint urn:nbn:de:0011-n-6378241 (2.0 MByte PDF)
MD5 Fingerprint: 5a0e41418276c6d11ffe1a1c586e730e
Erstellt am: 8.7.2021

Kurosu, M.:
Human-Computer Interaction. Design and User Experience Case Studies. Proceedings. Pt.III : Thematic Area, HCI 2021, Held as Part of the 23rd HCI International Conference, HCII 2021, Virtual Event, July 24-29, 2021
Cham: Springer Nature, 2021 (Lecture Notes in Computer Science 12764)
ISBN: 978-3-030-78467-6 (Print)
ISBN: 978-3-030-78468-3 (Online)
International Conference on Human-Computer Interaction (HCI International) <23, 2021, Online>
Bundesministerium fur Wirtschaft und Energie BMWi (Deutschland)
Innovationswettbewerb 2020; 01MK20011A; SPEAKER
Konferenzbeitrag, Elektronische Publikation
Fraunhofer IIS ()
user experience; Task Success; Voice User Interfaces; measuring; SUS; SASSI; SUISQ; AttrakDiff; Alexa; Siri; Google Assistant

Speech assistants exhibit a high error rate with about one in three user requests resulting in an error. Nonetheless, speech assistants are adopted rapidly with about 1.8 billion users expected in 2021. Given the relatively high task failure rate of speech assistants this may be surprising and raises the question how much user experience (UX) is affected by task success in these devices. We measure user experience with four metrics of UX and evaluate task success in interactions with the speech assistants Alexa, Google Assistant, and Siri. We find that task success only explains between 13% and 28% of the variance of UX. This suggests that a majority of UX is not explained by whether an assistant successfully completes tasks. Moreover, we find that the three assistants do not significantly differ in task success rate, but differ in UX, which supports the conclusion that task success and UX possess limited alignment. We discuss our results and point out limitations and potential future work.