Research map / Large language models / AI and society
Benchmarking XAI Explanations with Human-Aligned Evaluations
Rémi Kazmierczak, Steve Azzolin, Eloïse Berthier, Anna Hedström, Patricia Delhomme, David Filliat, Nicolas Bousquet, Goran Frehse, Massimiliano Mancini, Baptiste Caramiaux, Andrea Passerini, Gianni Franchi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SigLIP image encoder instead of BLIP for PASTA-score embeddings (ablation) | PASTA-score · PASTA-dataset Q1 | QWK: 0.328 -> 0.471 | Improved |
| CLIP vs SigLIP embeddings (ablation) | PASTA-score · PASTA-dataset Q1 | QWK: 0.450 -> 0.471 | No clear effect |
| automated PASTA-score versus human annotators | PASTA-score · PASTA-dataset Q1 | QWK: 0.471 -> 0.849 (human) | Hurt |
| select explainer per image by highest PASTA-score | ResNet50 explainer mixture · ResNet50 faithfulness correlation | faithfulness correlation: 0.0579 -> 0.0627 | Improved |
| increase CLIP backbone size from ViT-B to ViT-g | activation-map XAI explanations · PASTA-score | PASTA-score: 3.12 -> 2.75 (example) | Hurt |
| add PASTA-score regularizer to RISE (lambda=0.7) (ablation) | RISE · image 1 | PASTA-score: 4.13 -> 4.19 | Improved |
| add PASTA-score regularizer to RISE (lambda=0.7) (ablation) | RISE · image 1 | faithfulness correlation: 0.0983 -> 0.0874 | Hurt |
| set RISE lambda=1 (optimize PASTA-score only) (ablation) | RISE · image 1 | faithfulness correlation: 0.0983 -> 0.0326 | Hurt |
| add PASTA-score regularizer to RISE (lambda=0.7) (ablation) | RISE · image 2 | faithfulness correlation: 0.0692 -> 0.0731 | Improved |
Closest papers by meaning
- Navigating the Maze of Explainable AI: A Systematic Approach to Evaluating Methods and Metrics (NeurIPS 2024 · similarity 0.90)
- I-CEE: Tailoring Explanations of Image Classification Models to User Expertise (AAAI 2024 · similarity 0.88)
- Counterfactual Explanations for Misclassified Images: How Human and Machine Explanations Differ (Abstract Reprint) (AAAI 2024 · similarity 0.87)
- HumanPCR: Probing MLLM Capabilities in Diverse Human-Centric Scenes (ICLR 2026 · similarity 0.86)
- Towards Unifying Evaluation of Counterfactual Explanations: Leveraging Large Language Models for Human-Centric Assessments (AAAI 2025 · similarity 0.86)
- Distinguishing Performance From Competence in Evaluations of Humanlike Abstract Reasoning (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
