Research map / Large language models / AI and society
Position: Explain to Question not to Justify
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Position paper arguing that XAI is split into human-value-oriented BLUE XAI and model-validation-oriented RED XAI, and that RED XAI (questioning and debugging models) is under-explored and needs new methods, tools and benchmarks.
Problem
The XAI field is slowed by divergent and incompatible goals, with most work and surveys focused on human-value goals (trust, fairness, ethics) while model validation, exploration and debugging receive less attention.
Approach
Separates XAI into two cultures, BLUE XAI (lay-user, human values) and RED XAI (developer-facing model validation and debugging), and argues explanations should be used to question models, not only to justify decisions, outlining research challenges for RED XAI such as complementary explanations, benchmarks/tools, explorer mindset, Rashomon-set analysis and XAI for science.
Key results
- Reviews of highly cited 2023 XAI surveys are found to be dominated by BLUE XAI perspectives, while RED XAI is comparatively under-represented.
- Identifies RED XAI challenges including construction of complementary explanations, need for benchmarks/tools/standards, explorer mindset, Rashomon-set multi-model analysis, and XAI for science.
Surprising findings
- Argues that user studies should not be a universal requirement for XAI papers, since model-development explanations are not seen by end users.
Limitations
- Argument is qualitative and based on a non-systematic selection of recent surveys
- Two-culture dichotomy is acknowledged to not fit all applications
Experimental setup
Builds on: Breiman two cultures LIME SHAP Rashomon set
Closest papers by meaning
- Evaluating the Effectiveness of Explainable Artificial Intelligence Approaches (Student Abstract) (AAAI 2024 · similarity 0.90)
- Navigating the Maze of Explainable AI: A Systematic Approach to Evaluating Methods and Metrics (NeurIPS 2024 · similarity 0.89)
- Position: Explainable AI Cannot Advance Without Better User Studies (ICML 2025 · similarity 0.88)
- F-Fidelity: A Robust Framework for Faithfulness Evaluation of Explainable AI (ICLR 2025 · similarity 0.85)
- iLLuMinaTE: An LLM-XAI Framework Leveraging Social Science Explanation Theories Towards Actionable Student Performance Feedback (AAAI 2025 · similarity 0.85)
- Even-if Explanations: Formal Foundations, Priorities and Complexity (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.