Research map / Vision-language and multimodal / Multimodal LLMs

ViSurf: Visual Supervised-and-Reinforcement Fine-Tuning for Large Vision-and-Language Models

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ViSurf, a single-stage post-training method that injects ground-truth labels as high-reward samples into GRPO-style RLVR rollouts, with three reward-control strategies, and reports gains over SFT, RLVR, and SFT→RLVR on Qwen2.5-VL-7B across segmentation, GUI, anomaly, medical, and math tasks.

Problem

Post-training LVLMs with SFT gives sub-optimal generalization and forgetting, while RLVR is capped by the model's existing knowledge; the sequential SFT→RLVR pipeline is costly and suffers forgetting.

Approach

Include the ground-truth label as one more high-reward rollout in the GRPO group so that a single objective yields both an SFT-like gradient term and an RLVR term, stabilized by aligning label formatting with the policy, zeroing the thinking reward for labels, and smoothing the label reward when rollouts already match or beat it.

Key results

• ViSurf reaches 66.6 gIoU and 57.1 N-Acc on gRefCOCO val versus 42.8 gIoU / 0.0 N-Acc for RLVR and 65.0 gIoU / 52.1 N-Acc for SFT→RLVR.
• Average score across the eight benchmarks is 69.6 for ViSurf versus 65.4 for SFT→RLVR, 56.1 for RLVR, and 50.2 for the baseline.
• On ChartQA and DocVQA (not trained on), ViSurf scores 87.4 and 95.0, retaining baseline-level DocVQA while SFT drops to 89.6.

Surprising findings

• Pure RLVR underperforms the untrained baseline on RealIAD and gRefCOCO, attributed to noisy incorrect rollouts.
• Removing the detailed prompt format leaves RLVR at 0.0 gIoU on ReasonSeg while ViSurf still reaches 57.8 test gIoU.

Limitations

• Ground-truth labels are limited to final answers in this work; reasoning traces are not used.
• The RL codebase (VeRL) mainly supports the Qwen-VL series, limiting architectural coverage.
• Theoretical analysis relies on approximations (pi_theta ≈ pi_old, single-sample single-update) and the paper's gradient derivation is informal.

Experimental setup

Datasets: gRefCOCO Multi-objects-7K ReasonSeg OmniACT RealIAD ISIC2018 MathVista WeMath MathVision Polymath SceMQA Geometry3K

Benchmarks: gRefCOCO ReasonSeg OmniACT RealIAD ISIC2018 MathVista ChartQA DocVQA

Metrics: gIoU N-Acc Accuracy ROC_AUC Bbox_Acc

Baselines: SFT RLVR SFT→RLVR LISA GSVA SAM4MLLM SegZero VisionReasoner

Builds on: GRPO DAPO VisionReasoner VeRL

Closest papers by meaning

• Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning (NeurIPS 2024 · similarity 0.89)
• VisRef: Visual Refocusing while Thinking Improves Test-Time Scaling in Multi-Modal Large Reasoning Models (CVPR 2026 · similarity 0.88)
• SSL4RL: Revisiting Self-supervised Learning as Intrinsic Reward for Visual-Language Reasoning (ICML 2026 · similarity 0.88)
• TTRV: Test-Time Reinforcement Learning for Vision Language Models (CVPR 2026 · similarity 0.88)
• On Robustness and Chain-of-Thought Consistency of RL-Finetuned VLMs (ICML 2026 · similarity 0.88)
• CoVFT: Context-aware Visual Fine-tuning for Multimodal Large Language Models (CVPR 2026 · similarity 0.88)