Research map / Vision-language and multimodal / Multimodal LLMs

Start Small, Think Big: Curriculum-based Relative Policy Optimization for Visual Grounding

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that GRPO-trained chain-of-thought hurts visual grounding and proposes CuRPO, a curriculum GRPO that orders training samples by CoT length or gIoU reward, improving over Visual-RFT on RefCOCO-family and LISA.

Problem

RL-finetuned chain-of-thought reasoning can degrade visual grounding accuracy, and larger training sets do not reliably improve performance due to varying data difficulty.

Approach

Use average CoT length (and gIoU-based reward) as a difficulty signal to sort training examples and feed them to GRPO in easy-to-hard curriculum phases, while the reward uses scaled gIoU plus format reward.

Key results

• On RefCOCO val, CuRPO (Length) reaches 33.80 mAP vs 21.28 for Visual-RFT (+12.52).
• On LISA, CuRPO with 50 training examples reaches 37.4 mIoU, above Visual-RFT with 239 examples (34.4).
• On Qwen2-VL-7B, CuRPO (Length) achieves 52.66 mIoU vs 45.29 for Visual-RFT.

Surprising findings

• CoT-based GRPO training degrades grounding relative to direct-output training, and adding data does not consistently help.
• Pearson correlation between CoT length and reward is -0.4395, indicating longer CoTs are associated with lower reward.

Limitations

• Difficulty signals rely on CoTs generated by a pretrained VLM, and the generator is not clearly named.
• Gains over Visual-RFT on RefCOCO are reported with per-split differences, and the Random curriculum already gives most of the improvement, so the benefit of length/reward ordering is smaller on some splits.
• Analysis is limited to visual grounding tasks.

Experimental setup

Datasets: RefCOCO RefCOCO+ RefCOCOg LISA

Benchmarks: RefCOCO RefCOCO+ RefCOCOg LISA

Metrics: mAP mIoU

Baselines: Visual-RFT SFT OV-Seg GroundingDINO GroundedSAM Qwen2-VL-2B (pretrained)

Builds on: GRPO Visual-RFT Generalized IoU Chain-of-Thought prompting

Closest papers by meaning

• Grounded Reinforcement Learning for Visual Reasoning (NeurIPS 2025 · similarity 0.90)
• Revisiting the Necessity of Lengthy Chain-of-Thought in Vision-centric Reasoning Generalization (CVPR 2026 · similarity 0.89)
• iVGR: Internalizing Visually Grounded Reasoning for MLLMs with Reinforcement Learning (ICML 2026 · similarity 0.89)
• GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agents (NeurIPS 2025 · similarity 0.88)
• Delving into RL for Image Generation with CoT: A Study on DPO vs. GRPO (NeurIPS 2025 · similarity 0.88)
• Decomposed On-Policy Distillation for Vision-Language Reasoning: Steering Gradients for Visual Grounding (ICML 2026 · similarity 0.88)