Defect evolution in a product line environment

One mechanism used for monitoring the development of the Space Shuttle flight control software, in order to minimize any risks to the missions, is the independent verification and validation (IV&V) process. Using data provided by both the Shuttle software developer and the IV&V contractor, in this paper we describe the overall IV&V process as used on the Space Shuttle program and provide an analysis of the use of metrics to document and control this process over multiple releases of this software. Our findings reaffirm the value of IV&V, show the impact of IV&V on multiple releases of a large complex software system, and indicate that some of the traditional measures of defect detection and repair are not applicable in a multiple-release environment such as this one.