Research map / Large language models / Reasoning and chain of thought

DiffAdapt: Difficulty-Adaptive Reasoning for Token-Efficient LLM Inference

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DiffAdapt, which classifies question difficulty from an LLM's prefill hidden states with a small MLP probe and selects Easy/Normal/Hard inference configs (prompt, temperature, max tokens), saving up to 22.4% tokens without fine-tuning the LLM.

Problem

Reasoning LLMs generate long chains of thought uniformly regardless of problem difficulty, wasting tokens on easy problems (overthinking) while under-allocating on hard ones.

Approach

Observe a U-shaped generation-entropy pattern across difficulty, then train a lightweight MLP probe on frozen prefill hidden states to map each question to one of three inference strategies (Easy/Normal/Hard), each with its own prompt, temperature and max-token budget, with labels generated heuristically from sampled correctness and entropy.

Key results

• DiffAdapt reports up to 22.4% token savings versus the fixed Normal strategy on Qwen3-4B, versus -27.5% for DEER on the same model.
• On Qwen3-4B, DiffAdapt (default) averages 70.9 vs 68.5 for a 30%-data probe and 67.7 for a linear head across token budgets.
• End-to-end time on 40 OlympiadBench problems drops from 64 min (vLLM baseline) to 10 min with DiffAdapt.

Surprising findings

• Easy-labeled problems show high generation entropy despite high accuracy, the reverse of the naive expectation that easy problems yield confident outputs.
• DEER increases token usage relative to the fixed Normal strategy (-27.5% on Qwen3-4B, -53.3% on DS-R1-Qwen-7B).

Limitations

• Difficulty thresholds are set heuristically per model and may need recalibration across domains or sampling configurations.
• The probe uses only prefill hidden states and ignores generation-time dynamics.
• Aggressive budget reduction can harm accuracy on hard or error-prone cases.

Experimental setup

Datasets: DeepMath-103K

Benchmarks: GSM8K MATH500 AIME 2024 AIME 2025 OlympiadBench Minerva GPQA MMLU-Pro

Metrics: accuracy token savings wall-clock latency

Baselines: DEER Fixed Easy strategy Fixed Normal strategy Fixed Hard strategy ThinkLess

Builds on: DEER DeepMath-103K

Closest papers by meaning

• ARES: Multimodal Adaptive Reasoning via Difficulty-Aware Token-Level Entropy Shaping (ICLR 2026 · similarity 0.91)
• Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens (ICML 2026 · similarity 0.90)
• Think or Not? Exploring Thinking Efficiency in Large Reasoning Models via an Information-Theoretic Lens (NeurIPS 2025 · similarity 0.90)
• Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs (ICLR 2026 · similarity 0.90)
• Hint Tuning: Less Data Makes Better Reasoners (NeurIPS 2026 · similarity 0.90)
• THE PATH OF LEAST RESISTANCE: GUIDING LLM REASONING TRAJECTORIES WITH PREFIX CONSENSUS (ICLR 2026 · similarity 0.90)