Research map / Large language models / Reasoning and chain of thought

MM-HELIX: Boosting Multimodal Long-Chain Reflective Reasoning with Holistic Platform and Adaptive Hybrid Policy Optimization

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces MM-HELIX, a 1,260-item multimodal benchmark of 42 reflective reasoning puzzle tasks, the MM-HELIX-100K SERG-generated training set, and AHPO, a reward-gated hybrid of NLL on expert traces and GRPO that improves Qwen2.5-VL-7B on MM-HELIX and math/logic benchmarks.

Problem

Current MLLMs are weak at long-chain reflective reasoning (iterative thinking and backtracking) on visual tasks, and standard RL suffers from sparse rewards while SFT causes catastrophic forgetting.

Approach

Adaptive Hybrid Policy Optimization adds an NLL loss on expert CoT traces weighted by a gate that switches on only when the group's on-policy success rate falls below a threshold, combined with a GRPO-style objective in a single stage; expert data comes from SERG (rule-based scaffold refined by Qwen3-235B and verified by a rule-based verifier).

Key results

• AHPO on Qwen2.5-VL-7B reaches 24.9% on MM-HELIX (+18.6 over base) and 42.2 average on general math/logic (+5.7 over base 36.5).
• GPT-5 scores 58.1% on multimodal MM-HELIX, while the best open-source model Intern-S1-241B reaches 33.3%; Qwen2.5-VL-72B reaches 13.9%.
• Plain GRPO yields only +2.7 on MM-HELIX, while SFT reaches 23.8 in-domain but drops general average to 29.9.

Surprising findings

• SFT on expert data drops general reasoning average by 6.6 points despite large in-domain gains, indicating catastrophic forgetting.
• Text-only versions of the tasks score far higher than the multimodal versions (e.g., GPT-5 84.5% vs 58.1%).

Limitations

• Standard RL fails on hard tasks due to reward sparsity when the base model cannot solve them.
• Static expert guidance causes conflict with evolving policy and can degrade performance.

Experimental setup

Datasets: MM-HELIX-100K MMK12

Benchmarks: MM-HELIX MathVision MathVerse-V LogicVista WeMath

Metrics: accuracy

Baselines: GRPO SFT SFT&GRPO LUFFY GPT-5 Qwen2.5-VL-72B

Builds on: GRPO LUFFY DAPO

Closest papers by meaning

• SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learning (NeurIPS 2025 · similarity 0.91)
• MathSE: Improving Multimodal Mathematical Reasoning via Self-Evolving Iterative Reflection and Reward-Guided Fine-Tuning (AAAI 2026 · similarity 0.90)
• SRPO: Self-Reflective Policy Optimization for Long-Horizon Reasoning (ICML 2026 · similarity 0.89)
• Stop Unnecessary Reflection: Training LRMs for Efficient Reasoning with Adaptive Reflection and Length Coordinated Penalty (ICLR 2026 · similarity 0.89)
• Rectifying LLM Thought from Lens of Optimization (ICLR 2026 · similarity 0.89)
• R-4B: Incentivizing General-Purpose Auto-Thinking in MLLMs via Bi-Mode Annealing and Reinforce Learning (CVPR 2026 · similarity 0.89)