Research map / Large language models / Reasoning and chain of thought

On the Token Value Inequality in Efficient Reasoning

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Long CoT dynamics and efficiency · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TokenProbe, a token-level log-probability diagnostic (positive ratio of normalized log-probs) and two GRPO modifications (selective KL anchoring and reward shaping) that compress chain-of-thought redundancy while preserving accuracy.

Problem

Chain-of-thought reasoning consumes many more tokens than needed, and it is unclear which tokens in a reasoning trace are valuable versus redundant; response-level length penalties cannot distinguish them.

Approach

Uses within-response normalized token log-probabilities (NormLP) and their positive ratio (TokenProbe) as an inference-native signal to identify core tokens, then injects a core-token mask into GRPO via a per-token KL/sparsity term (Selective KL Anchoring) and advantage shaping (Reward Shaping) to compress redundant tokens.

Key results

• Qwen-8B w/ KL reaches 62.7% accuracy at 1,476 tokens versus Gemini-3.1-Pro at 54.4% and 1,586 tokens on the four math benchmarks average.
• Compresses response length by 1.8-4.2x across backbones with essentially no accuracy loss; Qwen-8B w/ RS raises accuracy to 68.3% from 61.9% while cutting tokens from 6,252 to 2,313.
• Qwen-0.6B w/ KL attains 42.3% at 2,023 tokens versus base 40.7% at 5,890 tokens, while PEAR and ETR give only 1.3-1.4x reduction.

Surprising findings

• Vanilla GRPO collapses Qwen3-4B-Instruct (non-thinking) from 71.7% to 1.3% accuracy, and the authors restrict the method to thinking-enabled backbones.
• Across experimental settings, TokenProbe correlates negatively with token length and positively with accuracy, yet the paper's own tables show a Pearson correlation of only about 0.2-0.4.

Limitations

• Confidence-correctness relationship can reverse after a change in training regime; signal depends on the generating policy.
• No compression headroom for non-thinking models; the method is restricted to thinking-enabled backbones.
• Out-of-domain validation limited to four QA benchmarks; code and agentic reasoning untested.

Experimental setup

Datasets: DeepScaleR-Preview AIME 2025 MATH-500 AMC OlympiadBench MMLU MMLU-Pro LSAT GPQA

Benchmarks: AIME 2025 MATH-500 AMC OlympiadBench MMLU MMLU-Pro LSAT GPQA

Metrics: accuracy average generated tokens

Baselines: DEER NoThink L1 O1-Pruner PEAR ETR Gemini-3.1-Pro GPT-5.4-Mini Claude-4.6-Sonnet DeepSeek-V3.2

Builds on: GRPO DeepScaleR DAPO Dr. GRPO

Closest papers by meaning

• Token Signature: Predicting Chain-of-Thought Gains with Token Decoding Feature in Large Language Models (ICML 2025 · similarity 0.91)
• Reasoning about Reasoning: BAPO Bounds on Chain-of-Thought Token Complexity in LLMs (ICML 2026 · similarity 0.90)
• On the Bias of Next-Token Predictors Toward Systematically Inefficient Reasoning: A Shortest-Path Case Study (NeurIPS 2025 · similarity 0.90)
• Reasoning Can Be Restored by Correcting a Few Decision Tokens (ICML 2026 · similarity 0.90)
• IAPO: Information-Aware Policy Optimization for Token-Efficient Reasoning (ICML 2026 · similarity 0.89)
• Training-Trajectory-Aware Token Selection (ICML 2026 · similarity 0.89)