Research map / Vision-language and multimodal / Visual question answering
Benchmarking Visual LLMs Resilience to Unanswerable Questions on Visually Rich Documents
Davide Napolitano, Luca Cagliero, Fabrizio Battiloro
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use higher corruption complexity 3 instead of 1 | Qwen2.5-VL 72B · DUDE | Acc_D: 0.588 -> 0.267 | Hurt |
| use higher corruption complexity 3 instead of 1 | Qwen2.5-VL 72B · MPDocVQA | Acc_D: 0.613 -> 0.576 | Hurt |
| use higher corruption complexity 3 instead of 1 | Gemma3 27B · DUDE | Acc_P: 0.810 -> 0.679 | Hurt |
| use higher corruption complexity 3 instead of 1 | Qwen2.5-VL 7B · DUDE | Acc_P: 0.843 -> 0.731 | Hurt |
| scale model size from Qwen2.5-VL 7B to 72B | Qwen2.5-VL · DUDE | Acc_D: 0.460 -> 0.599 | Improved |
| scale model size from Qwen2.5-VL 7B to 72B | Qwen2.5-VL · DUDE | Acc_P: 0.835 -> 0.754 | Hurt |
| scale model size from InternVL3 9B to 78B | InternVL3 · DUDE | Acc_D: 0.267 -> 0.326 | Improved |
| scale model size from InternVL3 9B to 78B | InternVL3 · MPDocVQA | Acc_D: 0.241 -> 0.219 | Hurt |
| explicitly state in the prompt that questions may be unanswerable (ablation) | VLLMs · DUDE | Acc_D | Improved |
| add OCR-extracted text to the input (ablation) | VLLMs · DUDE | Acc_D | Improved |
| combine unanswerability instruction with OCR text (ablation) | VLLMs · DUDE | Acc_D | Improved |
| increase the page window size (ablation) | VLLMs · DUDE | Acc_P | Hurt |
Closest papers by meaning
- Benchmarking Retrieval-Augmented Multimomal Generation for Document Question Answering (NeurIPS 2025 · similarity 0.90)
- FRAMES-VQA: Benchmarking Fine-Tuning Robustness across Multi-Modal Shifts in Visual Question Answering (CVPR 2025 · similarity 0.89)
- VTQA: Visual Text Question Answering via Entity Alignment and Cross-Media Reasoning (CVPR 2024 · similarity 0.89)
- When Open-Vocabulary Visual Question Answering Meets Causal Adapter: Benchmark and Approach (AAAI 2025 · similarity 0.88)
- NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samples (NeurIPS 2024 · similarity 0.88)
- GRAM: Global Reasoning for Multi-Page VQA (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
