Options
2021
Conference Paper
Titel
Success is not Final; Failure is not Fatal - Task Success and User Experience in Interactions with Alexa, Google Assistant and Siri
Abstract
Speech assistants exhibit a high error rate with about one in three user requests resulting in an error. Nonetheless, speech assistants are adopted rapidly with about 1.8 billion users expected in 2021. Given the relatively high task failure rate of speech assistants this may be surprising and raises the question how much user experience (UX) is affected by task success in these devices. We measure user experience with four metrics of UX and evaluate task success in interactions with the speech assistants Alexa, Google Assistant, and Siri. We find that task success only explains between 13% and 28% of the variance of UX. This suggests that a majority of UX is not explained by whether an assistant successfully completes tasks. Moreover, we find that the three assistants do not significantly differ in task success rate, but differ in UX, which supports the conclusion that task success and UX possess limited alignment. We discuss our results and point out limitations and potential future work.
Author(s)