Experiment on explicit modeling of change impact views: Experiment material and data

In 2011, we conducted an experiment at Fraunhofer IESE to evaluate which impact the explicit modeling of change impacts in architecture design has on the flexibility of the resulting software architecture. Our hypothesis is that there is a positive correlation. We set up an experiment with students of the TU Kaiserslautern, who did a practical course at Fraunhofer IESE, in cooperation with the industry partner John Deere. We operationalized the hypothesis into design tasks for two groups, one applying our change impact modeling technique during architecture design, the other not doing it. In the evaluation of the experiment data we found a significant difference between the two groups, which was also statically significant. The group applying the change impact modeling produced more flexible architectures. Additionally, we found a strong effect size. The main intention of this report is to provide full access to details of the experiment for other researchers and reviewers, upfront to publishing the PhD thesis. This report provides all the materials used for conducting the experiment, and the raw data of the evaluation. Its intention is also to serve as a basis of information, which can be referenced from scientific publications, in order to give readers the background information, which cannot be described in scientific papers due to limitations of space. Details about the architectural approach, the experiment design, and results of the experiment will be submitted to QoSA 2012 [NS12] and WICSA 2012 as scientific publications. The description of the experiment and the materials are also included in the PhD thesis [Naa12].