Architectural Mitigation of Control AI Risk Factors for Safe Human-Robot-Collaboration

Control functions based on artificial intelligence (AI) will be a central component to realize advanced applications, such as human-robot collaboration. A necessity for this is safety assurance, i.e., ensuring that a system that integrates AI functions will not pose a significant risk to humans in its environment. Though safety assurance for AI is an active research field, AI systems are still not sufficiently trustworthy and architectural safeguards are required to deploy them in safety-critical applications. However, work on safety architectures for control AI functions is currently sparse.
In this paper, we propose a novel methodology for AI risk factor management which results in mitigation measures for implementation in a generic AI safety architecture. We apply our proposal to systematically safeguard a control AI function based on deep reinforcement learning following the use case of human-robot collaboration. For evaluation, we implement a set of concrete mitigation measures and measure their efficacy in simulation. Our results indicate that the proposed methodology results in measures that are effective at safeguarding the control AI function, paving the way for integrating such functions into safety-critical systems.