Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Toward a Statistically Well-Grounded Evaluation of Listening Tests - Avoiding Pitfalls, Misuse, and Misconceptions

: Nagel, F.; Sporer, T.; Sedlmeier, P.

Audio Engineering Society -AES-:
128th AES Convention: "Make the right connections". Vol.1 : London, UK, May 22 - 25,2010
Red Hook, NY: Curran, 2010
Audio Engineering Society (AES Convention) <128, 2010, London>
Conference Paper
Fraunhofer IDMT ()
Fraunhofer IIS ()

Many recent publications in audio research present subjective evaluations of audio quality based on the Recommendation ITU-R BS.1534-1 (MUSHRA, MUltiple Stimuli with Hidden Reference and Anchor). This is a very welcome trend because it enables researchers to assess the implications of their developments. The evaluation of listening tests, however, sometimes sufers from an incomplete understanding of the underlying statistics. The present paper aims at identifying the causes for the pitfalls and misconceptions in MUSHRA evaluations. It exemplifes the impact of falsely used or even misused statistics. Subsequently, schemes for evaluating the listeners' judgments that are well-grounded on statistical considerations comprising an understanding of the concepts of statistical power and efect size are proposed.