Evaluation of XAI Methods in a FinTech Context

As humans, we automate more and more critical areas of our lives while using machine learning algorithms to make autonomous decisions. For example, these algorithms may approve or reject job applications/loans. To ensure the fairness and reliability of the decision-making process, a validation is required. The solution for explaining the decision process of ML models is Explainable Artificial Intelligence (XAI). In this paper, we evaluate four different XAI approaches - LIME, SHAP, CIU, and Integrated Gradients (IG) - based on the similarity of their explanations. We compare their feature importance values (FIV) and rank the approaches from the most trustworthy to the least trustworthy. This ranking can serve as a specific fidelity measure of the explanations provided by the XAI methods.