Research map / Vision-language and multimodal / Visual question answering
Super-CLEVR: A Virtual Benchmark To Diagnose Domain Robustness in Visual Reasoning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add probabilistic uncertainty reasoning to NSVQA (ablation) | NSVQA · Super-CLEVR visual complexity shifts | Relative Degrade: 17.48 -> 12.88 | Helped |
| Add probabilistic uncertainty reasoning to NSVQA (ablation) | NSVQA · Super-CLEVR question redundancy shifts | Relative Degrade: 1.72 -> 0.84 | Helped |
| Add probabilistic uncertainty reasoning to NSVQA (ablation) | NSVQA · Super-CLEVR concept distribution shifts | Relative Degrade: 20.92 -> 13.72 | Helped |
| Add probabilistic uncertainty reasoning to NSVQA (ablation) | NSVQA · Super-CLEVR concept compositionality shifts | Relative Degrade: 11.44 -> 7.00 | Helped |
| Use modular reasoning instead of non-modular architectures | VQA models · Super-CLEVR question redundancy shifts | Relative Degrade: less than 2% vs around 20% | Helped |
| Use non-modular models instead of modular reasoning methods | VQA models · Super-CLEVR visual complexity shifts | Relative Degrade: less than 10% vs more than 12% | Helped |
| Disentangle visual understanding from symbolic reasoning | NSVQA · Super-CLEVR concept distribution shifts | Relative Degrade: NSVQA 20.92; FiLM 28.46; mDETR 36.34; NSCL 37.44 | Helped |
| Remove redundant information progressively from questions | GQA models · GQA testdev | Accuracy drop: mDETR: 0 to -14.56; NSCL: 0 to -2.59; NSVQA: 0 to -7.02; P-NSVQA: 0 to -5.78 | Mixed |
Closest papers by meaning
- FRAMES-VQA: Benchmarking Fine-Tuning Robustness across Multi-Modal Shifts in Visual Question Answering (CVPR 2025 · similarity 0.91)
- Perception Matters: Detecting Perception Failures of VQA Models Using Metamorphic Testing (CVPR 2021 · similarity 0.91)
- How Transferable Are Reasoning Patterns in VQA? (CVPR 2021 · similarity 0.91)
- Domain-Robust VQA With Diverse Datasets and Methods but No Target Labels (CVPR 2021 · similarity 0.91)
- Neuro-Symbolic Visual Reasoning: Disentangling "Visual" from "Reasoning" (ICML 2020 · similarity 0.91)
- When Open-Vocabulary Visual Question Answering Meets Causal Adapter: Benchmark and Approach (AAAI 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.