Research map / Vision-language and multimodal / Multimodal LLMs
Dual-Stage Value-Guided Inference with Margin-Based Reward Adjustment for Fast and Faithful VLM Captioning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ViMaR, a two-stage value-guided inference framework for VLM captioning that trains a TD value head with a margin-based CLIP reward, cutting inference time over VisVM while reducing hallucinations and improving benchmark scores in self-training.
Problem
Inference-time search for VLM captioning is computationally expensive (re-scoring all candidates at each sentence step) and prone to low-confidence, hallucinated content.
Approach
A two-stage search: first select the highest-value full caption from N×K candidates with a TD-trained value head, then refine only under-grounded sentences; the value head is trained with a margin-based CLIP reward that penalizes similarity below a calibrated threshold tau.
Key results
- ViMaR achieves CHAIR_S 23.1 and CHAIR_I 4.1 versus 26.2 and 4.6 for VisVM-guided search, with MMHal 3.75 vs 3.30.
- ViMaR runs in 108 s per sample versus 462 s for VisVM and 668 s for BoN, reported as about 4.3x faster than VisVM.
- Human raters prefer ViMaR captions over VisVM in 64.0% of pairwise comparisons; the paper reports a 15.87% average relative improvement on benchmarks after self-training LLaVA-Next-7B.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use two-stage value-guided search with targeted segment refinement instead of VisVM-guided search | LLaVA-Next-Mistral-7B captioning · COCO Val2017 | CHAIR_S: 26.2 -> 23.1 | Helped |
| Use two-stage value-guided search with targeted segment refinement instead of VisVM-guided search | LLaVA-Next-Mistral-7B captioning · COCO Val2017 | MMHal: 3.30 -> 3.75 | Helped |
| Use ViMaR two-stage search instead of VisVM-guided search | LLaVA-Next-Mistral-7B captioning · COCO Val2017 | Avg. Approx Time: 462 s -> 108 s | Helped |
| Use ViMaR-guided rather than VisVM-guided captions for self-training | LLaVA-Next-Mistral-7B · Visual comprehension benchmarks (average) | Avg.: +11.08% -> +15.87% | Helped |
| Use ViMaR search rather than original decoding | LLaVA-OneVision-Qwen · MM-Vet | MM-Vet: 58.8 -> 60.5 | Helped |
| Use ViMaR search rather than original decoding | LLaVA-OneVision-Qwen · MMBench | MMBench: 81.7 -> 84.8 | Helped |
| Use ViMaR search rather than original decoding | LLaVA-OneVision-Qwen · MMMU | MMMU: 47.3 -> 49.4 | Helped |
| Use ViMaR search rather than original decoding | LLaVA-OneVision-Qwen · MathVista | MathVista: 56.1 -> 56.9 | Helped |
| Use ViMaR search rather than original decoding | LLaVA-OneVision-Qwen · LLaVA-W | LLaVA-W: 86.9 -> 88.5 | Helped |
| Use ViMaR search rather than original decoding | LLaVA-Next-Mistral-7B · COCO Train2017 caption preference | Human win rate vs VisVM: 64.0% | Helped |
Surprising findings
- The paper's own table reports greedy decoding and BoN can underperform on some benchmarks, and the margin-based value model is trained on CLIP-based rewards without external supervision.
Limitations
- Relies on CLIP similarity as the sole reward signal, which may be a weak proxy for grounding.
- Gains rely on GPT-4o and a small human study (300 pairs); the cross-model result is reported only for LLaVA-OneVision-Qwen2-7B with a single model.
- Paper text has inconsistencies (e.g., 4x vs 4.3x speedups, unclear tau calibration and Table 2 time ordering).
Experimental setup
Datasets: COCO 2017 LLaVA-150K CHAIR MMHal-Bench MM-Vet MMBench MMMU MathVista CVBench LLaVA-Wild MMStar
Benchmarks: CHAIR MMHal-Bench MM-Vet MMBench MMMU MathVista CVBench LLaVA-Wild MMStar
Metrics: CHAIR_S CHAIR_I MMHal score MMHal rate win rate average inference time (s) relative average improvement
Baselines: Greedy decoding Best-of-N CLIP-PRM search VisVM search CLIP-BoN GPT4-BoN
Builds on: VisVM temporal-difference learning beam search CLIP-PRM
Closest papers by meaning
- ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs (NeurIPS 2025 · similarity 0.89)
- CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning (ICLR 2026 · similarity 0.88)
- BalCapRL : A Balanced Framework for RL-Based MLLM Image Captioning (NeurIPS 2026 · similarity 0.88)
- VL-RewardBench: A Challenging Benchmark for Vision-Language Generative Reward Models (CVPR 2025 · similarity 0.87)
- Revisiting the Role of Language Priors in Vision-Language Models (ICML 2024 · similarity 0.87)
- Probing Visual Language Priors in VLMs (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.