• English
  • Deutsch
  • Log In
    Password Login
    Research Outputs
    Fundings & Projects
    Researchers
    Institutes
    Statistics
Repository logo
Fraunhofer-Gesellschaft
  1. Home
  2. Fraunhofer-Gesellschaft
  3. Konferenzschrift
  4. Concept-Guided LLM Agents for Human-AI Safety Codesign
 
  • Details
  • Full
Options
2024
Conference Paper
Title

Concept-Guided LLM Agents for Human-AI Safety Codesign

Abstract
Generative AI is increasingly important in software engineering, including safety engineering, where its use ensures that software does not cause harm to people. This also leads to high quality requirements for generative AI. Therefore, the simplistic use of Large Language Models (LLMs) alone will not meet these quality demands. It is crucial to develop more advanced and sophisticated approaches that can effectively address the complexities and safety concerns of software systems. Ultimately, humans must understand and take responsibility for the suggestions provided by generative AI to ensure system safety. To this end, we present an efficient, hybrid strategy to leverage LLMs for safety analysis and Human-AI codesign. In particular, we develop a customized LLM agent that uses elements of prompt engineering, heuristic reasoning, and retrieval-augmented generation to solve tasks associated with predefined safety concepts, in interaction with a system model graph. The reasoning is guided by a cascade of micro-decisions that help preserve structured information. We further suggest a graph verbalization which acts as an intermediate representation of the system model to facilitate LLM-graph interactions. Selected pairs of prompts and responses relevant for safety analytics illustrate our method for the use case of a simplified automated driving system.
Author(s)
Geissler, Florian
Fraunhofer-Institut für Kognitive Systeme IKS  
Roscher, Karsten  
Fraunhofer-Institut für Kognitive Systeme IKS  
Trapp, Mario  
Fraunhofer-Institut für Kognitive Systeme IKS  
Mainwork
AAAI Spring Symposium 2024. Proceedings  
Conference
Association for the Advancement of Artificial Intelligence (AAAI Spring Symposium) 2024  
DOI
10.1609/aaaiss.v3i1.31188
Additional full text version
Landing Page
Language
English
Fraunhofer-Institut für Kognitive Systeme IKS  
Fraunhofer Group
Fraunhofer-Verbund IUK-Technologie  
Keyword(s)
  • artificial intelligence

  • AI

  • generative AI

  • large language model

  • LLM

  • safe AI

  • safety

  • software engineering

  • safety analysis

  • Cookie settings
  • Imprint
  • Privacy policy
  • Api
  • Contact
© 2024