Research map / Vision-language and multimodal / Multimodal LLMs

DocVAL: Validated Chain-of-Thought Distillation for Grounded Document VQA

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DocVAL, a validated chain-of-thought distillation framework that trains a compact VLM (Gemma3-12B/4B) to answer document VQA questions and localize answers with boxes, using a rule-based validator for filtering and iterative refinement.

Problem

Compact VLMs answer document VQA questions but localize answers poorly; standard fine-tuning and distillation do not teach how localization decisions are made, and ANLS ignores spatial precision.

Approach

Distill spatial chain-of-thought traces from large teacher VLMs, filter them with a rule-based validator (VAL) that uses OCR/text detection only at training time, and iteratively refine the student with pixel-level corrective feedback so the final student is a pure VLM at inference.

Key results

• DocVAL (Gemma3-12B) reaches 91.4 ANLS and 82.4 mAP on DocVQA, vs. 84.6 ANLS for base Gemma3-12B and 88.7 ANLS / 50.1 mAP for ARIAL (27B + OCR).
• Removing VAL filtering (training on 102K unfiltered traces) drops DocVQA mAP to 63.7, versus 76.1 with binary filter and 82.4 with filter plus verifier feedback.
• DocVAL (Gemma3-4B) reaches 88.7 ANLS and 69.1 mAP on DocVQA, improving over base Gemma3-4B by +10.1 ANLS.

Surprising findings

• Training on unvalidated 102K examples gives lower mAP than using only 38K validated examples (63.7 vs 73.6 on DocVQA).
• Removing explicit teacher reasoning traces (direct box supervision) drops mAP by 17.0 points on DocVQA.

Limitations

• Evaluated only on five benchmarks of scanned forms, receipts, invoices and web documents; handwriting, degraded scans and chart-dominant pages are not covered.
• Validator depends on text detector quality (DB-ResNet), which weakens supervision on handwritten or degraded regions.
• Single-bbox output fails on multi-line answers spanning disconnected regions.

Experimental setup

Datasets: DocVQA VisualMRC FUNSD CORD SROIE

Benchmarks: DocVQA VisualMRC FUNSD CORD SROIE

Metrics: ANLS mAP@IoU[0.5:0.95] IoU@0.5 IoU@0.75

Baselines: ARIAL DocLayLLM LayoutLLM LayTextLLM DLaVA InternVL3.5 Qwen3-VL-8B-Thinking Llama-3.2-11B-Vision Gemma3-4B Gemma3-12B

Builds on: chain-of-thought prompting validation-driven learning rationale distillation

Closest papers by meaning

• DocScope: Benchmarking Verifiable Reasoning for Trustworthy Long-Document Understanding (NeurIPS 2026 · similarity 0.89)
• Marten: Visual Question Answering with Mask Generation for Multi-modal Document Understanding (CVPR 2025 · similarity 0.89)
• GRAM: Global Reasoning for Multi-Page VQA (CVPR 2024 · similarity 0.88)
• M3Grounder: Mask-Based Multi-Span and Multi-Granular Grounding for Document QA (CVPR 2026 · similarity 0.88)
• DocSeeker: Structured Visual Reasoning with Evidence Grounding for Long Document Understanding (CVPR 2026 · similarity 0.88)
• Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning (NeurIPS 2024 · similarity 0.88)