Evaluating a quality model for software product assessments - a case study

Background: Software quality models have been proposed as a means for describing the concept of quality. Most quality models take an abstract view on quality characteristics. Therefore, they are not able to integrate measurement tools and metrics for conducting quality assessments of real software systems. To solve this problem, we developed a quality meta-model defining the structure of quality models that are detailed enough to specify quality characteristics and their links to metrics and measurement tools. Aim: In this paper, we present our evaluation of this meta-model in terms of its usability for constructing quality models that are suitable for quality assessments of real software systems. Method: For conducting the study, we developed an initial "proof-of-concept" quality model on the basis of static code analysis tools. This quality model was used for conducting quality assessments of Java-based software systems. The results were analyzed regarding two criteria: (1) the diversification provided by the results and (2) the congruence of the results with an independently conducted expert-based evaluation of the systems. Results: While the difference in the assessment results between the various systems is rather small, a correlation with the expert evaluation could be proven. Furthermore, the study provided useful insights for further work and improvements. Conclusions: We conclude that quality models based on the Quamoco metamodel are, in principle, capable of being operationalized for the automated quality assessment of software systems.