Practical Challenges in AI Risk Classification and Assessment: Insights from an AI Sandbox Pilot

The EU AI Act mandates regulatory sandboxes in each member state to support AI system development under regulatory supervision. However, concrete guidance on their design remains limited. This paper presents lessons from KIRR Real, a sandbox initiative that conducted Legal Quick Checks for 15 AI systems across manufacturing, automation, and human resources. The requirements assessed were based on the AI Act and, where applicable, the Machinery Regulation. Across 15 organizations, we identify recurring challenges in risk classification, the interpretation of unclear concepts such as “self-evolving behavior” under the Machinery Regulation, technical implementation, documentation, and resource allocation, indicating that additional guidance is needed. Current guidance inadequately addresses common practices like fine-tuning pre-trained models, and established AI development practices prove insufficient for AI Act compliance. We derive recommendations for future sandboxes emphasizing early-stage engagement, low-threshold access, and integrated legal-technical expertise, contributing to safe and compliant AI deployment.