Research map / Large language models / AI and society
Navigating the Maze of Explainable AI: A Systematic Approach to Evaluating Methods and Metrics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces LATEC, a large-scale benchmark evaluating 17 saliency/attention XAI methods with 20 metrics across image, volume, and point cloud modalities (7,560 combinations), and proposes a more robust rank-based evaluation scheme.
Problem
Existing XAI evaluation studies are limited in scope (few methods, architectures, modalities, and metrics), and relying on one or few metrics risks selection bias and unreliable method rankings.
Approach
A large-scale benchmark crossing 17 XAI methods, 20 metrics, multiple architectures and datasets across three modalities, with a meta-evaluation of metric disagreement and an aggregate-then-rank evaluation scheme (median of standardized scores, then ranking, with mean/median/SD of ranks).
Key results
- Expected Gradients ranks consistently among the top methods for faithfulness and robustness on image data and has average metric disagreement.
- Attention methods (RA, RoA, LA) show very large SD between metrics, with faithfulness and robustness metrics ranking them either very high or very low.
- Across the benchmark, 326,790 saliency maps and 378,000 evaluation scores were computed over 7,560 combinations and released.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Expected Gradients instead of other XAI methods | LATEC XAI benchmark · Computer vision modalities | faithfulness ranking | Helped |
| Use Expected Gradients instead of other XAI methods | LATEC XAI benchmark · Computer vision modalities | robustness ranking | Helped |
| Apply LIME or Kernel SHAP to image or volume data rather than point cloud data | LIME and Kernel SHAP · Image and volume modalities | faithfulness | Hurt |
| Use CAM methods on volume data rather than image data | CAM methods · Volume versus image modality | faithfulness | Hurt |
| Use attention methods instead of attribution methods | XAI methods · Computer vision modalities | robustness | Helped |
| Use LRP Attention instead of raw attention | Transformer attention methods · Computer vision modalities | faithfulness | Helped |
| Use LRP Attention instead of LRP on Transformer architectures | Relevance-based XAI methods · Transformer architectures | faithfulness | Helped |
| Use LRP Attention instead of LRP on Transformer architectures | Relevance-based XAI methods · Transformer architectures | robustness | Helped |
| Apply SmoothGrad to the original XAI methods (ablation) | XAI methods in LATEC · Faithfulness | faithfulness | No clear effect |
| Apply SmoothGrad to the original XAI methods (ablation) | XAI methods in LATEC · Robustness | robustness | No clear effect |
| Apply SmoothGrad to the original XAI methods (ablation) | XAI methods in LATEC · Saliency map complexity | complexity | Helped |
| Cast saliency maps from 64-bit to 32-bit precision (ablation) | LATEC saliency maps · LATEC evaluation | evaluation stability | Hurt |
Surprising findings
- Expected Gradients, the top-performing method, was not evaluated in any of the 18 related studies.
- Complexity metrics rank CAM and attention methods as more complex, which the authors find counter-intuitive compared to the saliency maps.
Limitations
- Restricted to computer vision modalities (image, volume, point cloud); excludes language and video.
- Complexity metrics are questioned as not matching human understandability, so complexity rankings should be interpreted with caution.
- Benchmark focuses on comparing methods rather than absolute faithfulness or robustness of individual methods.
Experimental setup
Datasets: ImageNet UCSD OCT retina RESISC45 AdrenalMNIST3D OrganMNIST3D VesselMNIST3D CoMA ModelNet40 ShapeNet
Benchmarks: LATEC benchmark (17 XAI methods x 20 metrics x 3 modalities)
Metrics: faithfulness robustness complexity rank mean rank median rank standard deviation Levene test proportion Wilcoxon-Mann-Whitney p-value
Baselines: Occlusion LIME Kernel SHAP Vanilla Gradient Integrated Gradients Expected Gradients GradCAM Raw Attention
Builds on: Captum Quantus LRP Integrated Gradients Expected Gradients
Closest papers by meaning
- Position: Explain to Question not to Justify (ICML 2024 · similarity 0.89)
- F-Fidelity: A Robust Framework for Faithfulness Evaluation of Explainable AI (ICLR 2025 · similarity 0.88)
- Position: Explainable AI Cannot Advance Without Better User Studies (ICML 2025 · similarity 0.88)
- Evaluating the Effectiveness of Explainable Artificial Intelligence Approaches (Student Abstract) (AAAI 2024 · similarity 0.87)
- Even-if Explanations: Formal Foundations, Priorities and Complexity (AAAI 2025 · similarity 0.86)
- Evaluate with the Inverse: Efficient Approximation of Latent Explanation Quality Distribution (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.