Towards a comprehensive approach for assessing open source projects

Open Source Software (OSS) has an increasing importance for the software industry. Similar to traditional (closed) software acquisition, OSS acquisition requires an assessment of whether quality is sufficient for the intended purpose. This includes assessing a software component's intrinsic quality, as well as its supplier's maturity (i.e., ability to deliver high quality) and sustainability (i.e., whether the supplier will continue to exist). For traditional software acquisition, established procedures are available for evaluating these aspects. These procedures need to be adapted for OSS projects, because they have no traditional supplier, but an underlying OSS community. The openness of OSS development presents both challenges and opportunities for project evaluation. In particular, a variety of data sources are available that potentially allow for in-depth analysis, but it is not clear how to use them effectively. In this paper, we present an approach toward a comprehensive measurement framework for OSS projects, developed in the EU project QualOSS. This approach takes into account product quality as well as process maturity and sustainability of the underlying OSS community.