Research map / Vision-language and multimodal / Multimodal LLMs

Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Instruction-tuned multimodal assistants · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Vision-OPD, an on-policy self-distillation method where a model's crop-conditioned behavior (teacher) supervises its full-image behavior (student) via token-level JSD on student rollouts, improving fine-grained visual understanding without external teachers, labels, or verifiers.

Problem

MLLMs struggle with fine-grained visual understanding because small decisive evidence is overwhelmed by global context in full-image inference; existing fixes rely on costly inference-time zoom tools or sparse-reward RL and external teachers/labels.

Approach

Instantiate a crop-conditioned teacher and a full-image student from the same MLLM, sample rollouts from the student, and minimize per-token JSD between teacher and student distributions along those rollouts, with an EMA-regularized teacher; this internalizes zooming into a single forward pass.

Key results

• Vision-OPD-9B reaches 79.68 average across fine-grained benchmarks, above Gemini-3.1-Pro (79.25) and GPT-5.4 (72.77) in the reported table.
• Vision-OPD on Qwen3.5-9B improves V* Bench from 82.72 (vanilla) to 94.76 and ZoomBench from 52.07 to 65.80.
• On holdout tasks, Vision-OPD-9B keeps or improves performance (e.g., MMStar 83.07 to 83.20) while SFT on self-teacher drops MMStar to 73.33 and GRPO/DAPO also degrade holdout performance.

Surprising findings

• Using the current policy as teacher without regularization collapses training to near-zero accuracy.
• Training with the same model under crop input as teacher shows the regional-to-global gap of 18-22 points across models, including closed-source ones.

Limitations

• Relies on a 397B model to synthesize questions and labels for the training data and baselines.
• Teacher must be regularized (EMA) to avoid collapse; unregularized self-distillation fails.
• Evaluated only on Qwen3.5 backbones in non-thinking mode.

Experimental setup

Datasets: 6.2K synthetic triplets from unlabeled images via detection/segmentation and Qwen3.5-397B question/answer generation

Benchmarks: V* Bench ZoomBench HR-Bench 4K HR-Bench 8K MME-RealWorld EN MME-RealWorld CN MMVP CV-Bench MMStar POPE

Metrics: accuracy

Baselines: DeepEyes Thyme DeepEyesV2 SenseNova-MARS GPT-5.4 Gemini-3.1-Pro Qwen3.5-397B Kimi-K2.6 SFT on Self-Teacher GRPO DAPO OPSD

Builds on: on-policy distillation OPSD GRPO DAPO

Closest papers by meaning

• Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception (ICML 2026 · similarity 0.92)
• Seeing is Understanding: Unlocking Causal Attention into Modality-Mutual Attention for Multimodal LLMs (ICML 2026 · similarity 0.90)
• LION: Empowering Multimodal Large Language Model with Dual-Level Visual Knowledge (CVPR 2024 · similarity 0.90)
• Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs (CVPR 2024 · similarity 0.90)
• Seeing What Matters: A Training-Free Self-Guided Framework for Multimodal Detail Perception and Reasoning (CVPR 2026 · similarity 0.90)
• Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-training (ICLR 2026 · similarity 0.90)