Research map / Vision-language and multimodal / Multimodal LLMs

Enhancing Multi-Modal LLMs Reasoning via Difficulty-Aware Group Normalization

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Durian, which regroups GRPO/DAPO rollouts by perceptual difficulty (image patch-covariance eigenvalue entropy) and reasoning difficulty (mean sequence log-probability) so that samples in each group share a std for advantage normalization, improving Qwen2.5-VL-7B multimodal reasoning.

Problem

Std-based group normalization in GRPO is unstable for multimodal RLVR because groups with nearly all-correct or all-wrong rewards produce tiny std that over-amplifies extreme samples while neglecting balanced ones.

Approach

Instead of normalizing each prompt's rollouts by their own std, re-group samples into difficulty bins (by visual entropy and by model confidence) and share a std within each bin, then linearly combine the resulting advantages with the original GRPO advantage.

Key results

• Durian on DAPO with Geometry3K (2.1K) reaches average 59.3 vs 57.3 for vanilla DAPO and 56.9 for vanilla GRPO on five benchmarks.
• Durian (DAPO, ViRL39K 39K samples) reaches average 60.1, with MathVision 29.9 and WeMath 72.0.
• The paper reports an average 11.3% improvement over Qwen2.5-VL-7B-Instruct (53.3 average).

Surprising findings

• Reasoning-confidence regrouping alone gives a 3.8% gain on MathVerse that the authors note even surpasses their full method on that benchmark.
• Vanilla DAPO's gain over vanilla GRPO is modest (57.3 vs 56.9 average), so Durian's gains are largely on top of both.

Limitations

• Introduces several extra hyperparameters (number of groups, combination weights, percentile thresholds) that need tuning.
• Difficulty proxies (visual entropy, sequence log-probability) are validated only indirectly via error-rate analysis judged by Gemini2.5 Pro.
• Main experiments use a small 2.1K training set on a single base model (Qwen2.5-VL-7B).

Experimental setup

Datasets: Geometry3K ViRL39K

Benchmarks: MathVerse MathVision MathVista WeMath HallusionBench

Metrics: accuracy

Baselines: Vanilla GRPO Vanilla DAPO R1-VL-7B Vision-R1-7B R1-OneVision-7B OpenVLThinker-7B MM-Eureka-Qwen-7B ADORA-7B ThinkLite-7B-VL VLAA-Thinker-7B NoisyRollout GPT-4o

Builds on: GRPO DAPO PPO EasyR1

Closest papers by meaning

• SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal Reasoning (CVPR 2026 · similarity 0.89)
• Stable and Efficient Single-Rollout RL for Multimodal Reasoning (CVPR 2026 · similarity 0.89)
• Revisiting the Data Sampling in Multimodal Post-training from a Difficulty-Distinguish View (AAAI 2026 · similarity 0.88)
• Test-Time Matching: Unlocking Compositional Reasoning in Multimodal Models (ICLR 2026 · similarity 0.87)
• Perception-Aware Policy Optimization for Multimodal Reasoning (ICLR 2026 · similarity 0.87)