Fraunhofer-Gesellschaft

Publica

Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Toward a Statistically Well-Grounded Evaluation of Listening Tests - Avoiding Pitfalls, Misuse, and Misconceptions

 
: Nagel, F.; Sporer, T.; Sedlmeier, P.

Audio Engineering Society -AES-:
128th AES Convention: "Make the right connections". Vol.1 : London, UK, May 22 - 25,2010
Red Hook, NY: Curran, 2010
pp.299-308
AES Convention <128, 2010, London>
English
Conference Paper
Fraunhofer IDMT ()
Fraunhofer IIS ()

Abstract
Many recent publications in audio research present subjective evaluations of audio quality based on the Recommendation ITU-R BS.1534-1 (MUSHRA, MUltiple Stimuli with Hidden Reference and Anchor). This is a very welcome trend because it enables researchers to assess the implications of their developments. The evaluation of listening tests, however, sometimes sufers from an incomplete understanding of the underlying statistics. The present paper aims at identifying the causes for the pitfalls and misconceptions in MUSHRA evaluations. It exemplifes the impact of falsely used or even misused statistics. Subsequently, schemes for evaluating the listeners' judgments that are well-grounded on statistical considerations comprising an understanding of the concepts of statistical power and efect size are proposed.

: http://publica.fraunhofer.de/documents/N-219878.html