Research map / Vision-language and multimodal / Multimodal LLMs

Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Perception-R1, which adds a visual perception reward (judged by an LLM against textual visual annotations extracted from CoT trajectories) to GRPO-based RLVR, achieving strong multimodal math and general benchmark results with only 1,442 training samples.

Problem

Accuracy-only RLVR fails to improve multimodal perception in MLLMs (McNemar's test shows no significant perception gain), which bottlenecks multimodal reasoning; answer-based rewards are too sparse for perception.

Approach

Introduce a dense visual perception reward: textual visual annotations are extracted from correct CoT trajectories of a strong MLLM, and a judging LLM scores the fraction of annotations reflected in each policy response; this reward is added to format, accuracy, and repetition-penalty rewards under GRPO.

Key results

• Perception-R1-7B (1.4K samples) reaches 74.2 on MathVista, 54.3 on MathVerse, 72.0 on WeMath, and 60.8 on MMMU, outperforming Vision-R1-7B (200K data) on most benchmarks except EMMA.
• Applying the method to Qwen2.5-VL-7B raises MathVerse Vision-Only from 42.2 (base) and 47.1 (GRPO) to 50.1.
• Adding the visual perception reward to GRPO improves average score from 50.6 to 53.0 across 8 benchmarks.

Surprising findings

• Directly using Qwen2.5-VL-32B-IT as a reward model underperforms the proposed judged-annotation reward, which the authors attribute to reward hacking.
• A weak 7B judge leads to reward saturation and performance below the base model on MathVerse and MathVision, suggesting reward hacking.

Limitations

• Visual annotations depend on a proprietary teacher (Gemini-2.5-Pro) and on judge LLM quality; weak judges cause reward hacking.
• Training data is limited to geometry (Geometry3K), and generalization to other domains is only partially shown; Mulberry variant underperforms on math benchmarks.
• Some annotations contain reasoning information rather than pure visual information, which may introduce implicit reasoning distillation.

Experimental setup

Datasets: Geometry3K mulberry-260k

Benchmarks: MathVista MathVerse MathVision WeMath MMMU MMMU-Pro MMStar EMMA

Metrics: accuracy McNemar's test p-value one-sample t-test

Baselines: GRPO (accuracy-only) Vision-R1-7B MM-Eureka-7B VLAA-Thinker-7B SophiaVL-R1-7B OpenVLThinker-7B R1-OneVision-7B R1-VL-7B URSA-7B Qwen2.5-VL-72B-IT

Builds on: GRPO DeepSeek-R1 RLVR Geometry3K

Closest papers by meaning

• Perceptual-Evidence Anchored Reinforced Learning for Multimodal Reasoning (CVPR 2026 · similarity 0.94)
• Spotlight on Token Perception for Multimodal Reinforcement Learning (ICLR 2026 · similarity 0.94)
• Perception-Aware Policy Optimization for Multimodal Reasoning (ICLR 2026 · similarity 0.94)
• Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models (ICLR 2026 · similarity 0.93)
• Incentivizing Versatile Video Reasoning in MLLMs via Data-Efficient Reinforcement Learning (CVPR 2026 · similarity 0.92)
• Bad Seeing or Bad Thinking? Rewarding Perception for Multimodal Reasoning (ICML 2026 · similarity 0.92)