Does Understanding of Test Items Help or Hinder Subjective Assessment of Basic Audio Quality?

During listening tests for subjective evaluation of intermediate audio quality, sometimes test items in various foreign languages are presented. The perception of basic audio quality thereby may vary depending on the listeners' native language. This study investigated the role of understanding in quality assessment employing regular German sentences and sentences consisting of half-automatically generated German-sounding pseudo words. Especially less experienced listeners rated pseudo words slightly higher than German sentences of matching prosody. While references were heard longer for pseudo items, for other conditions they tend to hear German items longer. Though effects of understanding in our study were small, they may play a role in foreign languages that are less understandable than our pseudo sentences and differ in phoneme inventory.