AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

Teaching VLMs to Admit Uncertainty in OCR from Lossy Visual Inputs

Shuhao Guan, Moule Lin, Cheng Xu, Jinman Zhao, Derek Greene

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Bias-aware multi-image robustness

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Targets hallucination in VLM-based OCR on degraded document images by teaching models to express uncertainty rather than guess (abstract only; details unavailable).

Problem

VLMs replacing traditional OCR hallucinate fluent but incorrect text on visually degraded (lossy) document images without signaling uncertainty, which current accuracy-focused post-training encourages.

Approach

Post-training that trains VLMs to admit uncertainty in OCR outputs instead of guessing when visual input is lossy; the abstract does not specify the mechanism.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add GRPO with UNC reward after UNC-tag SFT (ablation)Qwen2.5-VL-7B · Blur-OCRtranscription accuracy (char-level): 0.830 -> 0.889Improved
Add GRPO after UNC-tag SFT (ablation)Qwen2.5-VL-7B · Blur-OCRUNC tag F1 (word-level): 0.286 -> 0.363Improved
Train SFT with UNC tags instead of plain transcripts (ablation)Qwen2.5-VL-7B · Blur-OCRtranscription accuracy (char-level): 0.868 -> 0.830Hurt
Add pseudo-labeled cold start before GRPO (ablation)Qwen2.5-VL-7B · Blur-OCRUNC tag F1 (word-level): 0.009 -> 0.574Improved
Use pseudo-labeled error-based cold start instead of random-tag cold start (ablation)Qwen2.5-VL-7B · Blur-OCRUNC tag F1 (word-level): 0.298 -> 0.363Improved
Use word-level reward with char-level cold start instead of char-level reward (ablation)Qwen2.5-VL-7B · Blur-OCRUNC tag F1 (word-level): 0.363 -> 0.574Improved
Use word-level cold start with word reward instead of char cold start (ablation)Qwen2.5-VL-7B · Blur-OCRUNC tag F1 (word-level): 0.574 -> 0.378Hurt
Lower F-beta beta from 1.0 to 0.5 (ablation)Qwen2.5-VL-7B · Blur-OCRUNC tag F1 (word-level): 0.574 -> 0.685Improved
Set lambda to 1.0 instead of 0.9 (ablation)Qwen2.5-VL-7B · Blur-OCRtranscription accuracy (word-level): 0.839 -> 0.767Hurt
Add tagging term to reward (lambda 0.9 vs 0.0) (ablation)Qwen2.5-VL-7B · Blur-OCRUNC tag F1 (word-level): 0.250 -> 0.685Improved
Add length-mismatch damping eta to reward (ablation)Qwen2.5-VL-7B · Blur-OCRtranscription accuracy (word-level): 0.782 -> 0.839Improved
Apply method to a different backbone (InternVL2.5-8B)InternVL2.5-8B · Blur-OCRUNC tag F1 (word-level): 0.622 vs best baseline 0.491Improved

Surprising findings

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy