Adapting multi-criteria decision analysis for assessing the quality of software products. Current approaches and future perspectives

Our great reliance on software-based systems and services nowadays requires software products of the highest quality. An essential prerequisite for developing software of guaranteed quality in a predictable way is the ability to model and objectively assess its quality throughout the project lifecycle. A potential approach must handle the abstract and multi-dimensional character of quality. Analogies between software quality assessment (SQA) and Multi-Criteria Decision Analysis (MCDA) motivated us to investigate how MCDA methods can support SQA. Thus we (1) identified the most relevant requirements for an SQA method, (2) reviewed existing SQA methods regarding potential benefits from using MCDA methods, and (3) assessed some popular MCDA methods regarding their applicability for SQA. We found that although a number of SQA methods proposed in recent years already adapt MCDA methods, the exact rationales for selecting a particular method are usually unclear or arbitrary. Usually, neither the goals nor the constraints of SQA are explicitly considered. Existing SQA methods do not meet the relevant requirements and mostly share the same weaknesses independent of whether they employ MCDA or not. In many cases, popular MCDA techniques are unsuitable for SQA because they do not meet its basic constraints, such as handling situations where data are scarce. We discuss the gaps identified in MCDA. The results led us to the conclusion that future research should focus on resolving the deficits of the existing SQA methods or of the most promising MCDA techniques rather than on inventing new methods from scratch.