CC BY 4.0Rheude, TillmannTillmannRheudeWirtz, AndreasAndreasWirtzKuijper, ArjanArjanKuijperWesarg, StefanStefanWesarg2024-10-092024-10-092024https://publica.fraunhofer.de/handle/publica/477064https://doi.org/10.24406/publica-373910.59275/j.melba.2024-ebd310.24406/publica-3739Convolutional neural networks (CNNs) achieve prevailing results in segmentation tasks nowadays and represent the state-of-the-art for image-based analysis. However, the under- standing of the accurate decision-making process of a CNN is rather unknown. The research area of explainable artificial intelligence (xAI) primarily revolves around understanding and interpreting this black-box behavior. One way of interpreting a CNN is the use of class ac- tivation maps (CAMs) that represent heatmaps to indicate the importance of image areas for the prediction of the CNN. For classification tasks, a variety of CAM algorithms exist. But for segmentation tasks, only one CAM algorithm for the interpretation of the output of a CNN exist. We propose a transfer between existing classification- and segmentation- based methods for more detailed, explainable, and consistent results which show salient pixels in semantic segmentation tasks. The resulting Seg-HiRes-Grad CAM is an exten- sion of the segmentation-based Seg-Grad CAM with the transfer to the classification-based HiRes CAM. Our method improves the previously-mentioned existing segmentation-based method by adjusting it to recently published classification-based methods. Especially for medical image segmentation, this transfer solves existing explainability disadvantages. The code is available at https://github.com/TillmannRheude/SegHiResGrad_CAMenBranche: HealthcareResearch Line: Computer vision (CV)Research Line: Machine learning (ML)LTA: Machine intelligence, algorithms, and data structures (incl. semantics)LTA: Generation, capture, processing, and output of images and 3D modelsDeep learningMedical image processingImage analysisjournal article