Research map / Vision-language and multimodal / Visual question answering
A Case Study of the Shortcut Effects in Visual Commonsense Reasoning
Keren Ye, Adriana Kovashka
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace person tags and pronouns in answer options with semantically equivalent references | TAB-VCR · VCR Q→A, RULE-SINGULAR | accuracy: 70.5 -> 65.4 | Hurt |
| Replace person tags and pronouns in answer options with semantically equivalent references | TAB-VCR · VCR QA→R, RULE-SINGULAR | accuracy: 72.4 -> 66.3 | Hurt |
| Mask the most influential answer token | VCR models · VCR Q→A, ADVTOP-1 | accuracy: STD VAL 63.8–69.9; ADVTOP-1 49.8–54.9 | Hurt |
| Train B2T2 with curriculum masking, initial ratio 0.30 and decay rate 5E-5 (ablation) | B2T2 · VCR Q→A, STD VAL | accuracy: 68.5 -> 69.9 | Improved |
| Train B2T2 with curriculum masking, initial ratio 0.30 and decay rate 5E-5 (ablation) | B2T2 · VCR Q→A, ADVTOP-1 | accuracy: 37.0 -> 54.5 | Improved |
| Train B2T2 with curriculum masking, initial ratio 0.30 and decay rate 5E-5 (ablation) | B2T2 · VCR QA→R, STD VAL | accuracy: 68.5 -> 70.6 | Improved |
| Use static random answer-token masking at probability 0.30 (ablation) | B2T2 · VCR Q→A, STD VAL | accuracy: 68.5 -> 64.1 | Hurt |
| Use static random answer-token masking at probability 0.30 (ablation) | B2T2 · VCR Q→A, ADVTOP-1 | accuracy: 37.0 -> 47.5 | Improved |
| Combine static answer-token masking at probability 0.10 with masked language modeling (ablation) | B2T2 · VCR Q→A, STD VAL | accuracy: 68.5 -> 69.1 | Improved |
| Augment training with rule-based modifications to person references (ablation) | B2T2 · VCR Q→A, RULE-SINGULAR | accuracy: 63.3 -> 78.8 | Improved |
| Augment training with rule-based modifications to person references (ablation) | B2T2 · VCR Q→A, ADVTOP-1 | accuracy: 37.0 -> 31.6 | Hurt |
| Augment training with adversarially masked answer tokens (ablation) | B2T2 · VCR Q→A, ADVTOP-1 | accuracy: 37.0 -> 81.4 | Improved |
Closest papers by meaning
- Not Another Text Benchmark: Putting the “Visual" Back in Visual Question Answering for Large Video Models (NeurIPS 2026 · similarity 0.90)
- Activating Visual Context and Commonsense Reasoning Through Masked Prediction in VLMs (AAAI 2026 · similarity 0.90)
- Commonsense Video Question Answering through Video-Grounded Entailment Tree Reasoning (CVPR 2025 · similarity 0.90)
- NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samples (NeurIPS 2024 · similarity 0.89)
- When Open-Vocabulary Visual Question Answering Meets Causal Adapter: Benchmark and Approach (AAAI 2025 · similarity 0.89)
- SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
