Deliverable D7.2 - Evaluation protocol and KPIs

This report contains the evaluation methodology for the IN-PREP project and its outputs, namely the progress of the MRPP and its impact on transboundary response planning. The methodology detailed below built the basis for the evaluations conducted in-line with the IN-PREP exercises. It was based on quantitative and qualitative assessments including the use of Critical Success Factors (CSFs) and Key Performance Indicators (KPI) as well as interviews and observations. These concepts are explained together with the theoretical basis for evaluation in section 2. The actual implementation of the evaluation is detailed in the section 3. The entire evaluation results across the 6 IN-PREP exercises will be detailed in Deliverable 7.5 ""Training Programme Evaluation"". Overall four different types of evaluations have been developed for IN-PREP: Project progress: In this area some high-level KPIs have been defined to be able to assess, if the IN-PREP team has delivered the key parts of the IN-PREP solution as promised in the DoA; Technical performance: The IN-PREP technical partners and the end-users of the MRPP will evaluate the performance of the MRPP using a defined set of technological and user experience related KPIs; Evaluation regarding the impact on transboundary response planning and training: The end-users of the IN-PREP solution and the ethical advisor will evaluate the non-technical parts of the IN-PREP solution (e.g. the handbook) using a set of non-technical KPIs; Planning and organisation of TTX/FSX: Additionally, the IN-PREP team members will evaluate the organization and performance of the TTX/FSX to be able to adapt and improve the planning and organization of the next exercise. Against this background it has to be stressed, that the WP7 related evaluations have particularly targeted the project progress, the user requirements (which are presented in D2.6), and the system requirements, which have been derived in WP3 and the IN-PREP Use-Cases (D2.7). The full list of KPIs is presented in the Annex. Ethical evaluations of the tools and exercises have been conducted under ask 2.2 (Legal, Ethical, and Privacy Impact Assessment).