Research map / Vision-language and multimodal / Multimodal LLMs
GridVQA-X: A Diagnostic Framework for Evaluating Multimodal Explainability Methods
Sujay Belsare, Sushant Kumar, Sudarshan Nikhil, Ponnurangam Kumaraguru, Chirag Agarwal
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train on D_spur (shortcut-injected data) instead of D_pure | MDETR · GridVQA D_pure evaluation | accuracy: 1.000 -> 49% | Hurt |
| evaluate M_spur on Depth-2 Mixed instead of Depth-1 attribute-only queries | M_spur · GridVQA D_pure | accuracy: 100% -> 8% | Hurt |
| increase query depth from 2 to 3 for M_spur on Mixed queries | M_spur · GridVQA D_pure Mixed | accuracy: 8% -> 14% | Mixed |
| increase query depth from 1 to 3 for EMAP on Mixed Form 0 | EMAP · GridVQA D_pure Mixed Form 0 | interaction fraction: 0.821 -> 0.673 | Hurt |
| increase query depth from 1 to 3 for InterSHAP on Mixed | InterSHAP · GridVQA D_pure Mixed | interaction score: 0.921 -> 0.629 | Hurt |
| switch task from Existence to Counting for InterSHAP | InterSHAP · GridVQA-X | interaction score: 0.742 -> 0.921 | Mixed |
| explain M_spur instead of M_pure with MultiSHAP | MultiSHAP · GridVQA-X | RMA: 0.616 -> 0.689 | Mixed |
| explain M_spur instead of M_pure with MultiViz | MultiViz · GridVQA-X | RMA: approx 0.44 for both | No clear effect |
| add adversarial distractors in D_pure rather than shortcut-consistent distractors (ablation) | Case-1 bag-of-words heuristic · GridVQA | shortcut success rate: 100% -> 36.07% | Hurt |
| add confuser-region distractors (Theorem 1) for dense depth-2 Mixed queries (ablation) | Partial logic heuristic · GridVQA D2_M_F0_d0.7 | predictive ratio: 16.40% and 16.87% | Hurt |
| add target-boost/suppress balancing to dataset generation (ablation) | Majority-class heuristic · GridVQA buckets | heuristic success rate: approx 44% | Hurt |
| increase density from d0.3 to d0.7 for M_spur on Depth-3 Mixed Form 0 | M_spur · GridVQA D_pure Mixed D3 Form 0 | accuracy: 14 -> 0 | Hurt |
Closest papers by meaning
- Measuring Cross-Modal Synergy: A Benchmark for VLM Explainability (NeurIPS 2026 · similarity 0.91)
- MMFineReason: Closing the Multimodal Reasoning Gap via Open Data-Centric Methods (NeurIPS 2026 · similarity 0.89)
- ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation (ICLR 2026 · similarity 0.88)
- CrossCheck-Bench: Diagnosing Compositional Failures in Multimodal Conflict Resolution (AAAI 2026 · similarity 0.88)
- Evaluating Cross-Modal Reasoning Ability and Problem Charactaristics with Multimodal Item Response Theory (ICLR 2026 · similarity 0.88)
- RPTS: Tree-Structured Reasoning Process Scoring for Faithful Multimodal Evaluation (AAAI 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
