Research map / Vision-language and multimodal / Visual question answering
Counterfactual Vision and Language Learning
Ehsan Abbasnejad, Damien Teney, Amin Parvaneh, Javen Shi, Anton van den Hengel
Check your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add counterfactual training on both questions and images | UpDn · VQA-CP test | Overall accuracy: 39.74 -> 42.12 | Improved |
| Add counterfactual training on both questions and images | UpDn · VQA-CP test | Other accuracy: 46.05 -> 48.34 | Improved |
| Intervene on questions only rather than both modalities (ablation) | UpDn with counterfactual training · VQA-CP test | Number accuracy: 11.93 -> 14.17 | Improved |
| Intervene on images only rather than both modalities (ablation) | UpDn with counterfactual training · VQA-CP test | Overall accuracy: 42.12 -> 41.01 | Hurt |
| Intervene on questions only rather than both modalities (ablation) | UpDn with counterfactual training · VQA-CP test | Overall accuracy: 42.12 -> 40.62 | Hurt |
| Add counterfactual training | Pythia · VQA v2 validation | Overall accuracy: 68.49 -> 68.77 | Improved |
| Add counterfactual training | GRU · Embodied QA, T−10 | dT: 0.74 -> 0.67 | Improved |
| Add counterfactual training | GRU · Embodied QA, T−50 | dΔ: 2.26 -> 2.52 | Improved |
| Use optimized counterfactual exogenous variables instead of posterior-only training (ablation) | LSTM with randomly initialized embeddings · SST | Accuracy: 85.21 -> 85.61 | Improved |
| Use optimized counterfactual exogenous variables instead of posterior-only training (ablation) | LSTM with trainable GloVe embeddings · SST | Accuracy: 87.24 -> 88.4 | Improved |
| Add counterfactual training | VGG-19 · CIFAR-10 | Accuracy: 95.04 -> 96.73 | Improved |
| Add counterfactual training | ResNet-101 · CIFAR-100 | Accuracy: 77.78 -> 80.0 | Improved |
Closest papers by meaning
- What If the TV Was Off? Examining Counterfactual Reasoning Abilities of Multi-modal Language Models (CVPR 2024 · similarity 0.90)
- Evidence-RL: Towards Evidence-intensive Visual Reasoning (NeurIPS 2026 · similarity 0.89)
- Generalizing from SIMPLE to HARD Visual Reasoning: Can We Mitigate Modality Imbalance in VLMs? (ICML 2025 · similarity 0.89)
- Consistency and Uncertainty: Identifying Unreliable Responses From Black-Box Vision-Language Models for Selective Visual Question Answering (CVPR 2024 · similarity 0.89)
- V-CECE: Visual Counterfactual Explanations via Conceptual Edits (NeurIPS 2025 · similarity 0.88)
- Escaping the SpuriVerse: Can Large Vision-Language Models Generalize Beyond Seen Spurious Correlations? (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
