Research map / Vision-language and multimodal / Visual question answering
GRAM: Global Reasoning for Multi-Page VQA
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends a pretrained single-page DocFormerv2 to multi-page DocVQA via interleaved page/document-token attention layers, a decaying doc-token attention bias, and an optional compression transformer (C-Former) for latency trade-offs.
Problem
Multi-page document visual question answering (MP-DocVQA) requires processing long sequences of many pages, but transformer self-attention scales quadratically and pretraining multi-page models is costly and data-scarce.
Approach
Interleave pretrained single-page encoder layers (applied per page with learnable doc tokens) with a slim document-level self-attention layer over only the doc tokens, so cross-page information flows with near-linear cost in page count; add a head-wise decaying bias favoring doc tokens and a compression transformer before decoding.
Key results
- GRAM (large) reaches 80.32 ANLS on MPDocVQA versus 76.40 for DocFormerv2concat with the same backbone size.
- GRAM (large) achieves 51.15 ANLS on DUDE, surpassing DocGptVQA (50.02) for state-of-the-art among the reported large models.
- The C-Former variant GRAM_C-Former trades some accuracy (77.60 on MPDocVQA) for latency and handles documents over 300 pages, where DocFormerv2concat runs out of memory at about 20 pages.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GRAM instead of DocFormerv2concat | DocFormerv2 base · MPDocVQA test | ANLS: 69.67 -> 73.68 | Helped |
| Use GRAM instead of DocFormerv2concat | DocFormerv2 large · MPDocVQA test | ANLS: 76.40 -> 80.32 | Helped |
| Use GRAM instead of DocFormerv2concat | DocFormerv2 large · DUDE test | ANLS: 48.44 -> 51.15 | Helped |
| Train on both MPDocVQA and DUDE rather than MPDocVQA only | GRAM large · DUDE test | ANLS: 51.15 -> 53.36 | Helped |
| Add 32 doc tokens rather than using no doc tokens (ablation) | GRAM · DUDE validation, all pages | ANLS: 46.16 -> 47.88 | Helped |
| Use decaying bias adaptation rather than no bias (ablation) | GRAM · DUDE validation, all pages | ANLS: +0.36% | Helped |
| Use decaying rather than constant bias adaptation (ablation) | GRAM · DUDE validation, all pages | ANLS: 46.14 -> 47.88 | Helped |
| Add page embeddings (ablation) | GRAM · DUDE validation, all pages | ANLS: 46.12 -> 47.88 | Helped |
| Replace page-based segmentation with fixed-length segments (ablation) | GRAM · DUDE validation, all pages | ANLS: 47.88 -> 45.22; 45.09; 44.39 | Hurt |
| Increase C-Former output length from 8 to 4096 (ablation) | GRAM with C-Former · DUDE validation, all pages | ANLS: 39.83 -> 43.59 | Helped |
| Add C-Former compression before decoding | GRAM large · MPDocVQA test | ANLS: 80.32 -> 77.60 | Hurt |
| Add C-Former compression before decoding | GRAM large · DUDE test | ANLS: 51.15 -> 45.47 | Hurt |
Surprising findings
- Using a constant doc-token attention bias hurt results on DUDE, while the head-wise decaying bias improved them.
Limitations
- Relies on a page as the atomic unit, which may not hold for documents without clear page structure.
- C-Former compression lowers accuracy relative to the full-length GRAM model.
- Experiments use DocFormerv2 with a limited 4-page training context and 800 tokens per page.
Experimental setup
Datasets: MPDocVQA DUDE DocVQA
Benchmarks: MPDocVQA DUDE
Metrics: ANLS
Baselines: Longformer BigBird LayoutLMv3 Hi-VT5 T5-2D DocGptVQA DocBlipVQA DocFormerv2concat
Builds on: DocFormerv2 T5 ALiBi Hi-VT5 Longformer
Closest papers by meaning
- Marten: Visual Question Answering with Mask Generation for Multi-modal Document Understanding (CVPR 2025 · similarity 0.89)
- DocVAL: Validated Chain-of-Thought Distillation for Grounded Document VQA (ICML 2026 · similarity 0.88)
- DocSeeker: Structured Visual Reasoning with Evidence Grounding for Long Document Understanding (CVPR 2026 · similarity 0.87)
- VTQA: Visual Text Question Answering via Entity Alignment and Cross-Media Reasoning (CVPR 2024 · similarity 0.87)
- Efficient Large Multi-modal Models via Visual Context Compression (NeurIPS 2024 · similarity 0.87)
- DocVXQA: Context-Aware Visual Explanations for Document Question Answering (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.