Research map / Vision-language and multimodal / Multimodal LLMs

DiG: Differential Grounding for Enhancing Fine-Grained Perception in Multimodal Large Language Models

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Instruction-tuned multimodal assistants

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DiG, a proxy task where MLLMs localize all differences between 3D-rendered image pairs, trained with curriculum GRPO on Qwen3-VL-Thinking models, reporting gains on perception, grounding, and general benchmarks.

Problem

MLLMs struggle with fine-grained visual perception and precise spatial reasoning, and existing RL perception approaches either overspecialize on grounding datasets or use proxy tasks not targeted at fine-grained perception.

Approach

Frame fine-grained perception as differential grounding: given two similar images with an unknown number of differences, the model outputs bounding boxes for all differences; training data come from a controllable Blender 3D rendering pipeline, optimized with GRPO using F1 plus IoU rewards and a single-to-mixed difficulty curriculum.

Key results

• On Qwen3-VL-8B-Thinking, DiG raises HalBench from 73.3 to 76.7 and V* from 79.1 to 81.2, averaging 80.5 vs 79.3 on perception benchmarks.
• On RefCOCO val@50 with Qwen3-VL-8B, DiG improves from 86.9 to 90.0.
• On Qwen3-VL-4B, DiG raises MME from 1592.4 to 1643.4 and MMStar from 66.7 to 70.2.

Surprising findings

• Gains on the RefCOCO grounding benchmarks transfer from a difference-localization proxy task that never sees referring expressions.
• The reward function's IoU term alone produces a noticeable performance drop relative to F1 alone, so spatial overlap is insufficient as a reward.

Limitations

• Training data is small (about 4.8K synthetic pairs) and limited to simple geometric primitives rendered in Blender, which may limit transfer to natural images.
• The paper reports mostly small absolute gains (often 1-3 points) without variance or significance analysis.
• Hardware and compute budget are not reported.

Experimental setup

Datasets: DiG synthetic dataset (Blender-rendered) RefCOCO RefCOCO+ RefCOCOg

Benchmarks: HalBench HRBench-8K POPE V* VSR CV-Bench MMVP RefCOCO RefCOCO+ RefCOCOg MMBench MM-Vet

Metrics: accuracy F1 IoU Acc@50 Acc@75 Acc@95

Baselines: Qwen3-VL-4B-Thinking Qwen3-VL-8B-Thinking Qwen2.5-VL-7B Qwen2.5-VL-72B InternVL2.5-78B LLaVA-NeXT InternVL3.5-38B

Builds on: GRPO EasyR1 Qwen3-VL Hungarian algorithm

Closest papers by meaning

• PGT: Procedurally Generated Tasks for improving visual grounding in MLLMs (ICML 2026 · similarity 0.91)
• GeM-VG: Towards Generalized Multi-image Visual Grounding with Multimodal Large Language Models (AAAI 2026 · similarity 0.91)
• Breaking the Regional Perception Bottleneck of Multimodal Large Language Models via External Reasoning Framework (CVPR 2026 · similarity 0.90)
• Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception (ICML 2026 · similarity 0.90)
• Perception Tokens Enhance Visual Reasoning in Multimodal Language Models (CVPR 2025 · similarity 0.90)
• GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluation (CVPR 2026 · similarity 0.89)