Improving the identification of hedonic quality in user requirements: A second controlled experiment

[Context and Motivation] Systematically engineering a good user experience (UX) into a computer-based system under development demands that the user requirements of the system reflect all needs, including emotional, of all stakeholders. User requirements address two different types of qualities: pragmatic qualities (PQs), that address system functionality and usability, and hedonic qualities (HQs) that address the stakeholder's psychological well-being. Studies show that users tend to describe such satisfying UXes mainly with PQs, and that some users seem to believe that they are describing an HQ when they are actually describing a PQ. [Question/Problem] The problem is to see if classification of any user requirement as PQ-related or HQ-related is difficult, and if so, why. [Principal/Ideas/Results] We conducted two controlled experiments involving the same twelve requirements-engineering and UX professionals, hereinafter called "analysts". The first experiment, which had the twelve analysts classifying each of 105 user requirements as PQ-related or HQ-related, shows that neither (1) an analyst's involvement in the project from which the requirements came nor (2) the analyst's use of a detailed model of the qualities in addition to the standard definitions of "PQ" and "HQ" has a positive effect on the consistency of the analyst's classification with that of others. The second experiment, which had the twelve analysts classifying each of a set of 50 user requirements, derived from the 105 of the first experiment, showed that difficulties seem to be caused both by the analyst's lacking skill in applying the definitions of "PQ" and "HQ" and by poorly written user requirement specifications. [Contribution] The first experiment revealed that classification of user requirements is a lot harder than initially assumed. The second experiment provided evidence that the difficulties can be mitigated by the combination of (1) training analysts in applying the definitions of "PQ" and "HQ" and (2) casting user requirement specifications in a new template that forces provision of the information needed for reliable classification. The experiment shows also that neither training analysts nor casting user requirement specifications in the new template, by itself mitigates the difficulty in classifying user requirements.