Research map / Large language models / Reasoning and chain of thought

e3: Learning to Explore Enables Extrapolation of Test-Time Compute for LLMs

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes e3, a recipe (asymmetric skill chaining, negative-gradient RL, and a coupled data-difficulty and token-budget curriculum) that trains Qwen3-1.7B to in-context explore and extrapolate test-time compute beyond its training budget, reaching state-of-the-art sub-2B results on AIME'25 and HMMT'25.

Problem

Reasoning LLMs trained with RL or SFT at a fixed output-token budget often fail to improve when test-time compute is extrapolated beyond that training budget (e.g., 2x to 32k tokens).

Approach

Extrapolation arises from learning in-context exploration, which requires (1) base-model asymmetries such as a verification-generation gap, (2) negative gradients in RL that push probability mass toward longer traces that chain new asymmetries, and (3) a coupled curriculum that pairs problem difficulty with the smallest RL-friendly token budget.

Key results

• e3-1.7B reaches pass@1 of 43.8% on AIME 2025 at k=1, versus 35.5% for Qwen3-1.7B, and 67.2% pass@32 versus 65.2% for the base model.
• On HMMT 2025, e3-1.7B achieves 56.1% pass@32 versus 54.9% for Qwen3-1.7B and 45.0% for Nemotron-Reasoning-1.5B.
• e3-1.7B, trained up to a 16k budget, extrapolates to 32k tokens and is reported to outperform the best other sub-2B model by more than 8% on AIME'25 peak performance.

Surprising findings

• Most existing open reasoning models plateau or gain little when test budget is extended beyond their training budget, including RL-trained models.
• Training on easy problems only gave the best out-of-distribution AIME'25 performance at 32k, despite AIME being harder than the training data.

Limitations

• Study limited to model scale (1.7B) and math/combinatorial domains.
• Repetition bias in the base model limits benefits of further output-length scaling.
• Curriculum design relies on heuristic budget selection (kappa=1.2) and fixed budget grid (4k, 8k, 16k).

Experimental setup

Datasets: DeepScaleR DMath Countdown AIME 2025 HMMT 2025

Benchmarks: AIME 2025 HMMT 2025 Countdown n-digit multiplication

Metrics: pass@1 pass@k accuracy at test budget

Baselines: Qwen3-1.7B DeepSeek-R1-Distill-Qwen-1.5B Nemotron-Reasoning-1.5B s1.1-32B OpenThinker-7B DeepSeek-R1-Distill-Qwen-7B

Builds on: GRPO DeepScaleR Verl s1 budget forcing

Closest papers by meaning

• Let's (not) just put things in Context: Test-time Training for Long-context LLMs (ICLR 2026 · similarity 0.92)
• Optimizing Test-Time Compute via Meta Reinforcement Finetuning (ICML 2025 · similarity 0.90)
• Provable Test-Time Scaling for Beam Search in LLM Reasoning (NeurIPS 2026 · similarity 0.89)
• LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling (NeurIPS 2026 · similarity 0.89)
• Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning (NeurIPS 2025 · similarity 0.89)
• HyPER: Bridging Exploration and Exploitation for Scalable LLM Reasoning with Hypothesis Path Expansion and Reduction (ICML 2026 · similarity 0.89)