Testing and inspecting reusable product line components: First empirical results

In recent years, product line development has increasingly received attention in industry as it enables software-developing organizations to reduce both cost and time of developing and maintaining increasingly complex systems as well as to address the demands for individually customized products. Successful product line development requires high quality of reusable artifacts in order to achieve the promised benefits. The unique issues of quality assurance in the context of systematic reuse, however, have not been quantitatively investigated so far. This paper describes a first empirical study comparing the two defect detection techniques, code inspections and functional testing, in the context of product line development. The primary goal of the study was to initially investigate the defect finding potential of the techniques on reusable software components with common and variant features. The major findings of the study are that the two techniques identified different types of defects on variants of a reusable component. Inspections are on average 66.39% more effective and need on average 36.84% less effort to detect a defect We found that both the testing and inspection techniques applied in the experiment were ineffective in identifying variant-specific defects. Overall, the results indicate that the standard quality assurance techniques seem to be insufficient to address special characteristics of reusable components.