A replicated experiment

Abstract
Inspection is considered a powerful method to detect defects in a software artifact. It is reported that savings are particularly high if inspections are used in early phases of the software development process, i.e. in the requirements definition phase. However, only few systematic techniques exist to support defect detection in requirements documents. One is perspective-based reading (PBR). The effectiveness of this technique has been validated in an experiment with software professionals at NASA/Goddard Space Flight Center. In this paper we describe a replication of this experiment within an academic environment to validate the original results. In the original experiment, no real team meetings were performed, i.e. the individual results were pooled into nominal team results. In contrast, in our replication effort we performed real team meetings, which allowed us to make a comparison between real and nominal teams. Moreover, we investigated how the technique supports detection of de fects belonging to different defect classes. The results are three-fold: (1) We basically confirm the results of the original experiment, that PBR helps to increase individual and team defect detection effectiveness compared to an Ad-hoc approach. (2) We found no statistically significant difference between real and nominal teams. (3) The analysis according to different defect classes only yield few statistically significant results due to the experimental setting. To document data collection and analysis, we used the Goal/Question/Metric approach. We found it highly beneficial for structuring data collection and analysis especially for replication purposes.