Using Multi-Modal LLMs to Create Models for Fault Diagnosis

Creating models that are usable for fault diagnosis is hard. This is especially true for cyber-physical systems that are subject to architectural changes and may need to be adapted to different product variants intermittently. We therefore can no longer rely on expert-defined and static models for many systems. Instead, models need to be created more cheaply and need to adapt to different circumstances. In this article we present a novel approach to create physical models for process industry systems using multi-modal large language models (i.e ChatGPT). We present a five-step prompting approach that uses a piping and instrumentation diagram (P&ID) and natural language prompts as its input. We show that we are able to generate physical models of three systems of a well-known benchmark. We further show that we are able to diagnose faults for all of these systems by using the Fault Diagnosis Toolbox. We found that while multi-modal large language models (MLLMs) are a promising method for automated model creation, they have significant drawbacks.