Research map / Large language models / Reasoning and chain of thought

PEAR: Phase Entropy Aware Reward for Efficient Reasoning

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PEAR, a GRPO reward that penalizes thinking-phase token entropy (with answer-phase entropy term) to shorten reasoning traces of LRMs while keeping accuracy.

Problem

Large reasoning models produce overly long chain-of-thought responses with redundant reasoning, inflating inference cost; existing length-reduction methods rely on curated concise data or explicit length targets.

Approach

Observes that entropy correlates positively with response length and is higher in the thinking phase than the answer phase; uses a phase-dependent entropy penalty (thinking entropy minus alpha times answer entropy) added to the RLVR reward so the model learns concise reasoning without length targets.

Key results

• On Qwen3-4B, PEAR reduces average tokens by 59.4% (8614 to 3498) with average accuracy 80.54 vs 81.06 original.
• On DeepSeek-R1-Distill-Qwen-1.5B, PEAR cuts average tokens by 37.8% (4853 to 3016) with average accuracy 64.62 vs 64.41.
• On Qwen3-8B, PEAR reduces tokens by 55.2% (7751 to 3476) with accuracy 83.11 vs 84.02; trained only on GSM8K, it generalizes OOD to MATH500, AIME24, AMC23.

Surprising findings

• Penalizing thinking-phase entropy while not penalizing answer-phase entropy works better; setting alpha=-1 (penalizing both phases) degrades accuracy.
• Filtering out up to 40% of high-entropy thinking tokens leaves accuracy stable or improved on Qwen3-4B and Qwen3-8B.

Limitations

• Accuracy on AIME24 for DeepSeek-R1-Distill-Qwen-1.5B drops from 26.66 to 23.33, and results are reported on single runs with small benchmarks (AIME24 has 30 problems, AMC23 has 40).
• Evaluated only on math reasoning; training uses only GSM8K.

Experimental setup

Datasets: GSM8K

Benchmarks: GSM8K MATH500 AIME24 AMC23

Metrics: Acc@1 generated tokens

Baselines: GRPO Step Entropy LCPO

Builds on: GRPO LCPO Step Entropy verl

Closest papers by meaning

• Think or Not? Exploring Thinking Efficiency in Large Reasoning Models via an Information-Theoretic Lens (NeurIPS 2025 · similarity 0.90)
• Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoning (NeurIPS 2025 · similarity 0.90)
• Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning (NeurIPS 2025 · similarity 0.90)
• TERMINATOR: Learning Optimal Exit Points for Early Stopping in Chain-of-Thought Reasoning (NeurIPS 2026 · similarity 0.89)
• ARES: Multimodal Adaptive Reasoning via Difficulty-Aware Token-Level Entropy Shaping (ICLR 2026 · similarity 0.89)
• Promoting Efficient Reasoning with Verifiable Stepwise Reward (AAAI 2026 · similarity 0.89)