A Semantic Framework for Evaluating Post-hoc Explanations in Link Prediction

Knowledge Graphs (KGs) are often noisy or incomplete, and Link Prediction (LP) methods, especially those based on black-box KG-Embeddings, are employed to predict missing facts. Pushed by the need for trust in inferred facts, many methods for LP explanation (LP-X) have been created. However, comparing them is still an open issue due to multiple existing protocols. To address this gap, we envision the design of an automated and unified evaluation framework for post-hoc LP-Xs that allows for a systematic and operationalized computation and comparison of LP explanations. To offer a pragmatic view of our proposition, we extend the Explanation Ontology (EO) by enriching it with evaluation-specific constructs, thus providing a shared semantic model (i.e., a structured knowledge representation such as an ontology) that unifies LP-X methods, evaluation dimensions, and associated metrics. The model could be further extended to broader XAI methods. As a proof-of-concept, we instantiate the proposed EO extension with LP-DIXIT, a user-aware algorithmic explanation evaluation method, demonstrating the ontology's ability to address the targeted problem. Furthermore, we draw a solution for exploiting the semantic model, besides for the annotation and retrieval of different evaluation approaches based on multiple dimensions, but also for automating/operationalizing LP-Xs, given interest dimensions. The paper offers a view towards the foundation for a unified evaluation of post-hoc LP-Xs, and drafts the ground for automated user-centric assessment workflows.