Research map / Vision-language and multimodal / Visual question answering
Perception Matters: Detecting Perception Failures of VQA Models Using Metamorphic Testing
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Oscar's detected object tags as anchor points to align questions and images | VQA models · VQA 2.0 val, perception consistency MRs | error rate: Oscar+ large: 34.94% Reversion; GridFeat+MoViE+: 68.77% Reversion | Helped |
| Train using both training and validation splits rather than only the training split | Pythia · VQA 2.0 val, Partition | error rate: 92.04% -> 97.92% | Hurt |
| Train using both training and validation splits rather than only the training split | Pythia · VQA 2.0 val, Reversion | error rate: 71.25% -> 73.15% | Helped |
| Apply Removal+ by removing a referenced object and checking the count change | BERT-like VQA models · VQA 2.0 val, Removal+ | error rate: 99.56% VisualBERT; 99.74% ViLBERT | Hurt |
| Use partition questions containing two existing objects or properties rather than two nonexistent ones | Oscar+ large · VQA 2.0 val, Partition | error rate: Objects: 50.36% -> 93.35%; Properties: 57.97% -> 88.92% | Hurt |
Closest papers by meaning
- SQuINTing at VQA Models: Introspecting VQA Models With Sub-Questions (CVPR 2020 · similarity 0.95)
- Making the v in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering (CVPR 2017 · similarity 0.93)
- Roses Are Red, Violets Are Blue... but Should VQA Expect Them To? (CVPR 2021 · similarity 0.93)
- Debiased Visual Question Answering from Feature and Sample Perspectives (NeurIPS 2021 · similarity 0.92)
- VTQA: Visual Text Question Answering via Entity Alignment and Cross-Media Reasoning (CVPR 2024 · similarity 0.91)
- When Open-Vocabulary Visual Question Answering Meets Causal Adapter: Benchmark and Approach (AAAI 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.