Research map / Vision-language and multimodal / Visual question answering
A Novel Framework for Robustness Analysis of Visual QA Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Append increasingly lower-ranked groups of three GBQD questions to each main question | VQA models evaluated with the proposed sparse-combination ranking · VQA test-dev, GBQD partitions 1–7 | accuracy: 60.16 -> 49.96 | Hurt |
| Use sparse-combination ranking instead of direct similarity metrics to construct GBQD | VQA robustness evaluation · VQA test-dev, GBQD partitions 1–7 | monotonicity of accuracy across noise levels | Helped |
| Use lower-ranked GBQD question groups as added noise | VQA models · VQA test-dev, GBQD partitions 1–7 | accuracy: 60.16 -> 45.57 | Hurt |
| Evaluate robustness using YNBQD rather than GBQD | VQA models · VQA test-dev, YNBQD versus GBQD | robustness score | Mixed |
| Use attention-based models rather than non-attentive models | VQA models · VQA test-dev, GBQD | robustness score | Helped |
| Use MUTAN without Attention instead of MUTAN with Attention | MUTAN · VQA test-dev, YNBQD | robustness score | Helped |
| Use BLEU, ROUGE, CIDEr, or METEOR ranking instead of sparse-combination ranking | VQA robustness evaluation · VQA test-dev, GBQD | accuracy trend across noise levels | Hurt |
Closest papers by meaning
- Human-Adversarial Visual Question Answering (NeurIPS 2021 · similarity 0.92)
- VQAttack: Transferable Adversarial Attacks on Visual Question Answering via Pre-trained Models (AAAI 2024 · similarity 0.92)
- FRAMES-VQA: Benchmarking Fine-Tuning Robustness across Multi-Modal Shifts in Visual Question Answering (CVPR 2025 · similarity 0.90)
- NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samples (NeurIPS 2024 · similarity 0.89)
- When Open-Vocabulary Visual Question Answering Meets Causal Adapter: Benchmark and Approach (AAAI 2025 · similarity 0.88)
- On the Robustness of Large Multimodal Models Against Image Adversarial Attacks (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.