Research map / Vision-language and multimodal / Multimodal LLMs

M3Grounder: Mask-Based Multi-Span and Multi-Granular Grounding for Document QA

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes M3Grounder, a VLM plus promptable segmentation (SAM-style) model that emits [GROUND] tokens to produce hierarchical phrase/line/block masks for document QA grounding, and releases GroundingDocQA (200K docs, 2M QA) and GroundingDocQA-Bench.

Problem

Document QA systems answer text but often ground answers with coarse bounding boxes that fail on curved text and multi-span evidence; existing grounded DocVQA datasets are limited to extractive, box-level annotations.

Approach

Decouple answer generation from spatial prediction: the VLM interleaves [GROUND] tokens after answer spans, whose hidden states are projected into granularity-specific prompts that a SAM-style mask decoder turns into phrase, line, and block masks, regularized by a bleed-suppression loss and a hierarchical enclosure loss.

Key results

• M3Grounder-Q (Qwen3-VL-8B backbone) reaches 79.0 phrase-level F1g on GroundingDocQA-Bench, rising to 87.5 at block level, versus 60.6 for fine-tuned Qwen3-VL.
• On DOGR-Bench, M3Grounder-Q attains overall grounding F1 of 73.3, exceeding DOGR by 6.9 points at the same 8B scale.
• Hierarchical supervision ablation drops GroundingDocQA-Bench phrase-level F1g from 79.0 to 71.3 when trained only at phrase level.

Surprising findings

• LoRA-only adaptation substantially underperforms full fine-tuning, with the largest drop at phrase level.

Limitations

• Mask-based grounding increases computational cost relative to box-level methods.
• Formulation operates at single-page level; multi-page or long-context settings are not handled.

Experimental setup

Datasets: GroundingDocQA DocVQA InfographicVQA DeepForm KLC WTQ TabFact ChartQA

Benchmarks: GroundingDocQA-Bench BoundingDocs (test) DOGR-Bench MMDocBench DocVQA InfographicVQA ChartQA

Metrics: F1g (IoU>0.5) G-Eval answer quality EM F1txt IoU

Baselines: GPT-4o GPT-5 Gemini-2.5-Pro Gemma-3 Kimi VL mPLUG-DocOwl2 InternVL3.5 Qwen3-VL Kosmos 2.5 Chat DOGR

Builds on: SAM DOGR

Closest papers by meaning

• DocVAL: Validated Chain-of-Thought Distillation for Grounded Document VQA (ICML 2026 · similarity 0.88)
• DocScope: Benchmarking Verifiable Reasoning for Trustworthy Long-Document Understanding (NeurIPS 2026 · similarity 0.86)
• Benchmarking Retrieval-Augmented Multimomal Generation for Document Question Answering (NeurIPS 2025 · similarity 0.86)
• Docopilot: Improving Multimodal Models for Document-Level Understanding (CVPR 2025 · similarity 0.84)
• GRAM: Global Reasoning for Multi-Page VQA (CVPR 2024 · similarity 0.84)
• Marten: Visual Question Answering with Mask Generation for Multi-modal Document Understanding (CVPR 2025 · similarity 0.84)