Research map / Vision-language and multimodal / Visual question answering
Improving Selective Visual Question Answering by Learning From Your Peers
Corentin Dancette, Spencer Whitehead, Rishabh Maheshwary, Ramakrishna Vedantam, Stefan Scherer, Xinlei Chen, Matthieu Cord, Marcus Rohrbach
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use peer-generated correctness targets to train the Selector on A+B rather than self-generated targets | CLIP-ViL · VQA v2 test | coverage at 1% risk: 6.45 -> 18.40 | Improved |
| Use peer-generated correctness targets to train the Selector on A+B rather than self-generated targets | OFA-Base · VQA v2 test | coverage at 1% risk: 26.64 -> 27.71 | Improved |
| Use peer-generated correctness targets to train the Selector on A+B rather than self-generated targets | OFA-Large · VQA v2 test | coverage at 1% risk: 31.47 -> 32.92 | Improved |
| Train a Selector instead of using the model's MaxProb confidence | OFA-Large · VQA v2 test | coverage at 1% risk: 16.31 -> 31.47 | Improved |
| Use peer-generated correctness targets to train the Selector on A+B rather than self-generated targets | CLIP-ViL · 90% VQA v2 + 10% AdVQA | coverage at 1% risk: 2.97 -> 15.22 | Improved |
| Use peer-generated correctness targets to train the Selector on A+B rather than self-generated targets | OFA-Base · 90% VQA v2 + 10% AdVQA | coverage at 1% risk: 19.72 -> 21.58 | Improved |
| Use peer-generated correctness targets to train the Selector on A+B rather than self-generated targets | OFA-Large · 90% VQA v2 + 10% AdVQA | coverage at 1% risk: 22.68 -> 25.38 | Improved |
| Add KNN OOD-detection features to the Selector (ablation) | OFA-Base Selector · 90% VQA v2 + 10% AdVQA | coverage at 1% risk: 19.05 -> 19.92 | No clear effect |
| Add SSD OOD-detection features to the Selector (ablation) | OFA-Base Selector · 90% VQA v2 + 10% AdVQA | coverage at 1% risk: 19.05 -> 18.99 | No clear effect |
| Add known OOD training examples from OK-VQA and VizWiz to Selector training (ablation) | OFA-Base Selector · 90% VQA v2 + 10% AdVQA | coverage at 1% risk: 19.00 -> 18.48 | Hurt |
| Reduce peer splits from 10 to 2 (ablation) | OFA-Base LYP · VQA v2 test | coverage at 1% risk: 27.71 -> 27.64 | No clear effect |
| Reduce Selector training data from 100% to 5% of A+B (ablation) | OFA-Base LYP · VQA v2 test | coverage at 1% risk: 27.71 -> 22.62 | Hurt |
Closest papers by meaning
- Reliable Visual Question Answering: Abstain Rather Than Answer Incorrectly (ECCV 2022 · similarity 0.94)
- Contrast and Classify: Training Robust VQA Models (ICCV 2021 · similarity 0.91)
- Roses Are Red, Violets Are Blue... but Should VQA Expect Them To? (CVPR 2021 · similarity 0.91)
- Consistency and Uncertainty: Identifying Unreliable Responses From Black-Box Vision-Language Models for Selective Visual Question Answering (CVPR 2024 · similarity 0.90)
- SIEVES: Selective Prediction Generalizes through Visual Evidence Scoring (NeurIPS 2026 · similarity 0.89)
- When Open-Vocabulary Visual Question Answering Meets Causal Adapter: Benchmark and Approach (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
