CC BY 4.0Sudhi, VijuVijuSudhiBhat, Sinchana RamakanthSinchana RamakanthBhatRudat, MaxMaxRudatTeucher, RomanRomanTeucherFlores-Herr, NicolasNicolasFlores-Herr2025-11-122025-11-122025-09https://publica.fraunhofer.de/handle/publica/499131https://doi.org/10.24406/publica-625110.48550/arXiv.2509.0762010.24406/publica-6251arXiv:2509.07620v1Retrieval Augmented Generation (RAG) systems, despite their growing popularity for enhancing model response reliability, often struggle with trustworthiness and explainability. In this work, we present a novel, holistic, model-agnostic, post-hoc explanation framework leveraging perturbation-based techniques to explain the retrieval and generation processes in a RAG system. We propose different strategies to evaluate these explanations and discuss the sufficiency of model-agnostic explanations in RAG systems. With this work, we further aim to catalyze a collaborative effort to build reliable and explainable RAG systems.enInformation RetrievalExplainabilityRetrieval Augmented GenerationLarge Language ModelsLLMspaper