Research map / Large language models / Reasoning and chain of thought

Learn to Reason Efficiently with Adaptive Length-based Reward Shaping

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LASER and its dynamic, difficulty-aware variants (LASER-D, LASER-DE) that reward correct responses for staying under adaptive target lengths, unifying truncation and other length-reward RL methods to compress reasoning traces of DeepSeek-R1-Distill models.

Problem

Large reasoning models produce redundant, over-long chains of thought, and existing RL efficiency methods trade substantial accuracy for token savings or are unstable or hackable.

Approach

A unified length-based reward shaping view covering truncation, group-based and budget-based rewards; LASER uses a step reward that gives a bonus to correct responses under a target length, and LASER-D makes the target length dynamic per difficulty bucket, chosen automatically via an expected-correct-responses (ECR) monitor.

Key results

• LASER-D (L_T=4096) on DeepSeek-R1-Distill-Qwen-1.5B reaches 60.3% average accuracy with 3,520 tokens vs. 56.9% and 10,177 tokens for the original model; AIME24 accuracy 34.2% vs. 28.9%.
• LASER-DE on the 1.5B model reaches 35.0% AIME24 accuracy with about 5,800 tokens, a 63% token reduction relative to the original on AIME.
• On DeepSeek-R1-Distill-Qwen-7B, LASER-D reaches 58.3% on AIME (+5.2) using 5,379 tokens vs. 13,414 for the original.

Surprising findings

• Length-penalty RL can improve accuracy on AIME24 while cutting tokens, unlike plain truncation which degrades hard-task accuracy.
• Group-based Efficient Reasoning reward shows reward hacking: accuracy drops while total reward rises, especially on simpler questions.

Limitations

• Evaluation limited to math tasks; generalization to code and agentic tasks untested.
• 32B experiments were run with a single configuration (L_T=8192 for LASER-DE) due to compute, and the training accuracy was already high (~76%), limiting headroom.
• Hyperparameters such as L_T and alpha are swept or chosen per model size, and the method introduces several design choices (buckets, ECR, K-sample monitoring).

Experimental setup

Datasets: DeepScaleR-Preview-Dataset MATH500 AIME 2024 AMC 2023 OlympiadBench GPQA LSAT MMLU

Benchmarks: MATH500 AIME2024 AMC2023 OlympiadBench GPQA LSAT MMLU

Metrics: accuracy (%) average generation length (tokens) keyword frequency reasoning behavior ratio

Baselines: Truncation ThinkPrune Efficient Reasoning (group-based reward) L1-Max Kimi-k1.5 Original DeepSeek-R1-Distill-Qwen

Builds on: GRPO ThinkPrune L1 Efficient Reasoning DAPO

Closest papers by meaning

• LEAD: Length-Efficient Adaptive and Dynamic Reasoning for Large Language Models (NeurIPS 2026 · similarity 0.92)
• DeepCompress: A Dual Reward Strategy for Dynamically Exploring and Compressing Reasoning Chains (ICLR 2026 · similarity 0.91)
• Resource-Efficient Reinforcement for Reasoning Large Language Models via Dynamic One-Shot Policy Refinement (ICML 2026 · similarity 0.91)
• RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs (ICLR 2026 · similarity 0.90)
• How Far Are We from Optimal Reasoning Efficiency? (NeurIPS 2025 · similarity 0.90)
• Promoting Efficient Reasoning with Verifiable Stepwise Reward (AAAI 2026 · similarity 0.90)