Research map / Vision-language and multimodal / Multimodal LLMs

Clarification as Supervision: Reinforcement Learning for Vision-Language Interfaces

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Trains a VLM captioner with RL using a tiered reward (1 for direct success, 0.7 for success after clarification, 0 for failure) so captions front-load information a frozen text reasoner needs, evaluated single-pass on seven visual math benchmarks.

Problem

Vision-language models trained to caption for humans omit precise details that text-only reasoners need for visual math, creating an interface mismatch; binary task rewards give sparse learning signals for learning captions suited to a specific reasoner.

Approach

Use clarification requests during training as implicit signals of information gaps: successes that require clarification from a frozen reference captioner earn partial reward (0.7), pressuring the trainable captioner to produce self-sufficient initial captions for single-pass inference, optimized with BNPO.

Key results

• AC-RL with Qwen2.5-VL-3B captioner raises seven-benchmark average from 39.0 to 43.4 (+4.4) over the Pretrained + Reasoner baseline.
• On DynaMath, AC-RL improves over Binary-Reward RL from 17.56 to 24.75 (+7.2).
• Clarification attempt rate drops from 49.57% to 30.28% on MathVerse_MINI (39% reduction) and from 40.69% to 28.95% on MathVision.

Surprising findings

• AC-RL shows a negative clarification gap on MathVerse (-2.54), meaning clarification slightly hurt accuracy for the trained captioner.
• Binary-Reward RL gives only modest gains over the decoupled baseline (+1.3 average), while AC-RL adds further gains.

Limitations

• Gains are modest on some benchmarks (e.g., minor regressions on WeMath) and the absolute accuracy remains well below frontier proprietary models.
• Evaluation restricted to mathematical visual reasoning; generalization to other reasoning domains is untested.
• Clarification reward depends on a frozen reference captioner and a fixed reasoner, so learned captions are tied to that specific reasoner.

Experimental setup

Datasets: ViRL-39K

Benchmarks: MathVista (testmini) MathVision MathVerse MMMU (validation) WeMath (strict) DynaMath (worst-case) LogicVista

Metrics: exact-match accuracy average accuracy clarification attempt rate clarification gap

Baselines: Binary-Reward RL (RACRO-style) Pretrained + Reasoner Standalone VLM GPT-4o Gemini 2.5 Pro InternVL3-78B-MPO QVQ-72B-Preview VL-Rethinker-72B

Builds on: RACRO BNPO GRPO DeepSeek-R1

Closest papers by meaning

• Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for Visual Reasoning (NeurIPS 2025 · similarity 0.89)
• More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.89)
• MindOmni: Unleashing Reasoning Generation in Vision Language Models with RGPO (NeurIPS 2025 · similarity 0.89)
• MiCo: Multi-image Contrast for Reinforcement Visual Reasoning (NeurIPS 2025 · similarity 0.89)
• Activating Visual Context and Commonsense Reasoning Through Masked Prediction in VLMs (AAAI 2026 · similarity 0.89)
• ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs (NeurIPS 2025 · similarity 0.89)