Research map / Vision-language and multimodal / Visual question answering
Multimodal Explanations: Justifying Decisions and Pointing to the Evidence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add visual attention to textual justification generation (ablation) | PJ-X · VQA-X | CIDEr: 66.3 -> 73.4 | Helped |
| add visual attention to textual justification generation (ablation) | PJ-X · ACT-X | CIDEr: 33.3 -> 58.7 | Helped |
| train on explanations rather than descriptions (ablation) | PJ-X · VQA-X | CIDEr: 36.2 -> 73.4 | Helped |
| train on explanations rather than descriptions (ablation) | PJ-X · ACT-X | CIDEr: 20.3 -> 58.7 | Helped |
| condition on predicted answer instead of ground-truth answer (ablation) | PJ-X · VQA-X | CIDEr: 73.4 -> 71.3 | Hurt |
| condition on predicted answer instead of ground-truth answer (ablation) | PJ-X · ACT-X | CIDEr: 58.7 -> 22.0 | Hurt |
| use textual-explanation supervision to learn visual pointing | PJ-X · VQA-X | Earth Mover's Distance: 2.77 -> 2.64 | Helped |
| use textual-explanation supervision to learn visual pointing | PJ-X · ACT-X | Earth Mover's Distance: 4.78 -> 2.54 | Helped |
| use textual-explanation supervision to learn visual pointing | PJ-X · VQA-X | Rank Correlation: +0.2211 -> +0.3423 | Helped |
| use textual-explanation supervision to learn visual pointing | PJ-X · ACT-X | Rank Correlation: +0.0104 -> +0.3933 | Helped |
| provide PJ-X explanations instead of no explanation | human diagnostic judgments · VQA-X | accuracy: 57.5% -> 70.0% | Helped |
| provide PJ-X explanations instead of no explanation | human diagnostic judgments · ACT-X | accuracy: 51.5% -> 80.5% | Helped |
Closest papers by meaning
- Enhancing Cognition and Explainability of Multimodal Foundation Models with Self-Synthesized Data (ICLR 2025 · similarity 0.89)
- Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoning (ICML 2026 · similarity 0.88)
- LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models (ICLR 2026 · similarity 0.87)
- DeFacto: Counterfactual Thinking with Images for Enforcing Evidence-Grounded and Faithful Reasoning (ICML 2026 · similarity 0.87)
- VLM-R³: Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thought (NeurIPS 2025 · similarity 0.87)
- Flex-Judge: Text-Only Reasoning Unleashes Zero-Shot Multimodal Evaluators (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.