Explainable Artificial Intelligence (xAI) for Object Detection Error Analysis in Automated Driving (AD)

Accurate object detection is crucial for the safety of automated driving (AD), as false environmental perception of Automated Vehicles (AV) can lead to accidents with significant risk to human life. Deep learning (DL)-based object detection models demonstrate outstanding performance in object detection tasks, comparable to that of humans, making their use state of the art (SOTA) in the perception of AVs. However, the complex and opaque architecture of DL-based object detectors complicates the understanding of detection error causes and subsequently challenges the safety of AD. One solution approach is the use of saliency map-based explainable artificial intelligence (XAI), which aims to visually highlight the input features deemed significant for the decision-making process of object detectors. While various XAI-based techniques for creating saliency maps exist, their use for quantitative feature detection and analysis, to our knowledge, has never been researched before. The thesis aims to pioneer this field by introducing two concepts that quantitatively generate saliency maps and accumulate highlighted features. The goal is to establish whether saliency-map-based XAI methods can help identify potential error causes in DL-based object detectors. The first concept generates saliency maps directly from the workings of object detector models. In the second concept, the error-making behaviour of object detectors is approximated with a meta-model, and saliency maps are generated from the meta-model’s decision process. In both concepts, features highlighted in the saliency maps are detected with the help of the Large Language Model (LLM) Gemini. Later post-processing then aggregates and hierarchically classifies the features into feature maps. Both concepts were implemented, and a comparative analysis of feature maps was conducted on five different error types to demonstrate the feasibility of both approaches in AD scenarios. The results demonstrate that the meta-model-based approach has limited usability in the thesis-created scenario. Error types are not presented in sufficient quantity for effective meta-model training, which leads to the model memorizing the training data without understanding underlying error causes. In contrast, the first concept of direct object detector-based saliency map generation is not limited in its functioning by smaller error sample sizes. The reliability of the generated feature distributions, however increased with
the number of error samples representing each error type. The comparative analysis of feature maps created from erroneous and non-error samples was able to determine features of interest with significantly different occurrences in both feature maps. Exemplary, manual assessment of the determined features from one of the error types led to the discovery of a concrete error cause. The thesis therefore successfully demonstrates that saliency map-based feature detection approaches can produce cues to error causes by narrowing down the search to potentially error-prone features. The final determination of whether and how the features are causing detection errors, however, was not feasible with saliency map-based approaches alone and required further in-depth feature analysis.