Research map / Large language models / Reasoning and chain of thought

Learning to Discover at Test Time

ICML 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TTT-Discover, which runs reinforcement learning at test time on a single scientific discovery problem using an entropic objective with adaptive beta and PUCT-based state reuse, and reports new state-of-the-art results on math bounds, GPU kernels, AtCoder contests, and single-cell denoising with gpt-oss-120b.

Problem

Existing test-time search for scientific discovery (e.g., AlphaEvolve) prompts a frozen LLM and never improves the policy, while standard RL optimizes average reward rather than the single best solution found for one hard problem.

Approach

Continue training the LLM with RL on the single test problem using an entropic utility objective (adaptive beta per initial state via a KL budget) that favors maximum-reward outcomes, combined with a PUCT-inspired rule over an archive of past solutions that uses max child reward for Q(s) and rank-based priors.

Key results

• Erdős minimum overlap upper bound improved to 0.380876, beating AlphaEvolve's 0.380924 with gpt-oss-120b.
• TriMul H100 kernel runtime 1161.2 us vs best human 1371.1 us; A100 2198.2 us vs human 4531.5 us.
• Denoising score on PBMC 0.71 vs MAGIC 0.64 and OpenEvolve 0.70; ahc058 score 848,414,228 beats top human 847,674,723.

Surprising findings

• Best-of-25600 baseline alone improved upon the AlphaEvolve Erdős construction.
• Ablation shows that removing the entropic objective or PUCT reuse leaves kernel runtime far worse (about 1985-2060 us vs 1203 us).

Limitations

• Only applicable to problems with continuous verifiable rewards; sparse or binary or non-verifiable domains are not handled.
• Second autocorrelation inequality: no discovery made (0.959 vs AlphaEvolve 0.961).
• Context window limit (32,768 tokens) truncates many OpenEvolve rollouts, so baseline comparison may be affected.

Experimental setup

Datasets: Pancreas (OpenProblems denoising) AtCoder ahc039 and ahc058 local tests GPUMode TriMul and MLA-Decode benchmarks

Benchmarks: Erdős minimum overlap first autocorrelation inequality second autocorrelation inequality circle packing GPUMode TriMul GPUMode MLA-Decode AtCoder ahc039 AtCoder ahc058 OpenProblems denoising (PBMC, Tabula Muris Senis Lung)

Metrics: upper bound (lower is better) lower bound (higher is better) runtime in microseconds contest score denoising score, MSE, Poisson NLL

Baselines: AlphaEvolve AlphaEvolve V2 ThetaEvolve OpenEvolve ShinkaEvolve ALE-Agent MAGIC ALRA Best-of-25600 naive test-time RL

Builds on: AlphaEvolve GRPO PUCT OpenEvolve Tinker

Closest papers by meaning

• Scaling Test-Time Compute Without Verification or RL is Suboptimal (ICML 2025 · similarity 0.89)
• Optimizing Test-Time Compute via Meta Reinforcement Finetuning (ICML 2025 · similarity 0.89)
• TTRL: Test-Time Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
• ThetaEvolve: Test-time Learning on Open Problems (ICML 2026 · similarity 0.87)
• LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling (NeurIPS 2026 · similarity 0.87)
• e3: Learning to Explore Enables Extrapolation of Test-Time Compute for LLMs (ICLR 2026 · similarity 0.86)