Navigating the Landscape of AI Test Methods Using Taxonomy-Based Selection

Due to broad deployment of AI systems in risk-prone domains and AI regulations coming into effect, systematic risk-and quality assessments of AI systems have become increasingly important. Conducting such assessments involves identifying relevant quality criteria for a given AI system and selecting test methods, i.e., procedures for collecting and evaluating evidences and measurable quantities that fit the identified criteria. This selection process can be challenging due to the high complexity of the test method landscape and, in the context of independent assessments, due to potential conflicts of interest between the involved stakeholders. To address this challenge, we present a practical solution approach for systematic, taxonomy-based selection of test methods. The paper closes with an outline of the gaps and possible next steps for utilizing test methods to achieve scalable and comparable, independent AI assessments.