Research map / Large language models / Reasoning and chain of thought

Stop Unnecessary Reflection: Training LRMs for Efficient Reasoning with Adaptive Reflection and Length Coordinated Penalty

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ARLCP, an RL method that penalizes over-reflection adaptively by problem complexity plus a length penalty, cutting response length by 53% (1.5B) and 35% (7B) while raising math accuracy.

Problem

Large reasoning models generate over-long chains of thought with excessive, unproductive reflection ("wait", "hmm"), which raises token cost and latency and often lowers accuracy, especially for smaller models and harder problems.

Approach

Estimate per-problem complexity from reflection token counts of correct rollouts, and apply a complexity-dependent reflection penalty plus a length penalty within a shared penalty budget (alpha1 + alpha2 = alpha) in an RLOO-trained reward that multiplies accuracy.

Key results

• On DeepSeek-R1-Distill-Qwen-1.5B, ARLCP improves overall accuracy by 5.81 points while cutting average length by 53.05%.
• On DeepSeek-R1-Distill-Qwen-7B, ARLCP gains 2.69 accuracy points with a 34.96% length reduction.
• On Qwen3-1.7B, ARLCP gives +3.00 accuracy and -38.19% length; on DeepSeek-R1-Distill-Llama-8B, +1.99 accuracy and -34.01% length.

Surprising findings

• GRPO collapses with the length-penalized reward (-2.71 accuracy, -65.2% length), so the authors switched to RLOO.
• Removing the reflection penalty yields the largest length reduction (-58.1%) but only +1.4 accuracy, versus +5.8 with both penalties.

Limitations

• Reflection is measured by fixed keyword matching, which may miss or misclassify reflective behavior.
• Complexity thresholds (n1, n2) are tuned on a pilot study and must be re-estimated for new models.
• Evaluation is mostly on math benchmarks; MMLU generalization is a single out-of-domain test.

Experimental setup

Datasets: DeepScaleR

Benchmarks: AMC 2023 AIME 2024 AIME 2025 GSM8K MATH-500 MMLU

Metrics: pass@1 accuracy response length (tokens) delta accuracy delta length (%)

Baselines: NoThinking SFT_Shortest DPO_Shortest O1-Pruner TLMRE AdaptThink LASER

Builds on: DeepScaleR RLOO VeRL DPO PPO GRPO REINFORCE++ AdaptThink LASER O1-Pruner

Closest papers by meaning

• CyclicReflex: Improving Reasoning Models via Cyclical Reflection Token Scheduling (ICLR 2026 · similarity 0.91)
• REA-RL: Reflection-Aware Online Reinforcement Learning for Efficient Reasoning (ICLR 2026 · similarity 0.91)
• How Far Are We from Optimal Reasoning Efficiency? (NeurIPS 2025 · similarity 0.91)
• Efficient Reasoning for Large Reasoning Language Models via Certainty-Guided Reflection Suppression (AAAI 2026 · similarity 0.91)
• Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models (NeurIPS 2025 · similarity 0.90)
• Learn to Reason Efficiently with Adaptive Length-based Reward Shaping (ICLR 2026 · similarity 0.90)