Fraunhofer-Gesellschaft

Publica

Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Interrater agreement in SPICE-based assesments

Some preliminary results
 
: Emam, K. el; Goldenson, D.R.; Briand, L.C.; Marshall, P.

:
Fulltext urn:nbn:de:0011-px-511929 (102 KByte PDF)
MD5 Fingerprint: 282514a75a6a56082c4c7b3540af829d
Created on: 10.08.2000


Kaiserslautern, 1997
IESE-Report, 043.97/E
Reportnr.: IESE-Report 043.97/E
English
Report, Electronic Publication
Fraunhofer IESE ()

Abstract
The international SPICE Project intends to deliver an ISO standard on software process assessment. This project is unique in software engineering standards in that there is a set of empirical trials, the objectives of which are to evaluate the prospective standard and provide feedback before standardization. One of the enduring issues being evaluated during the trials is the reliability of assessments based on SPICE. One element of reliability is the extent to which different teams assessing the same processes produce similar ratings when presented with the same evidence. In this paper we present some preliminary results from two assessments conducted during the SPICE trials. In each of these assessments two independent teams performed the same ratings. The results indicate that in general there is at least moderate agreement between the two teams in both cases. When we take into account the severity of disagreement then the extent of agreement between the two teams is almost perfect. Also, our results indicated that interrater agreement is not the same for different SPICE processes. The findings reported in this paper provide guidance for future studies of interrater agreement in the SPICE trials and also indicate some potential issues that need to be considered within the prospective standard.

: http://publica.fraunhofer.de/documents/PX-51192.html