Research map / Large language models / Reasoning and chain of thought

ThetaEvolve: Test-time Learning on Open Problems

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces ThetaEvolve, an open-source AlphaEvolve-style framework that uses a single small LLM with large program databases, batch sampling, and lazy penalties, and trains it with GRPO at test time on open optimization problems.

Problem

AlphaEvolve-style program evolution on open math optimization problems relies on closed frontier LLM ensembles and is inference-only, so the model cannot internalize evolving strategies; this paper asks whether small open models can reach new best-known bounds with test-time RL.

Approach

Simplifies AlphaEvolve to a single LLM with batched sampling and a large program database, adds early checks and a lazy penalty that discourages repeating known programs, and uses the evolution loop as a dynamic verifiable environment for GRPO-based test-time RL with optional reward shaping.

Key results

• Distill-Qwen3-8B with ThetaEvolve RL reaches 2.6359857 on CirclePacking-T, beating AlphaEvolve's 2.63586276 (even after shrinking radii for the strict task).
• ProRL-1.5B-v2 with RL reaches mean 2.3498 on CirclePacking-T vs 2.1313 for OpenEvolve at 307.2k programs and 2.0991 for ThetaEvolve without RL.
• Format-reward RL on ThirdAutoCorrIneq gives mean 1.6783, worse than pure inference (1.6766) and RL with the evaluator (1.6412).

Surprising findings

• A format-reward RL baseline performs worse than pure inference, indicating the gains come from evaluator-grounded learning rather than format learning.
• RL in a static environment (always starting from the initial program) performs much worse than even pure inference with the dynamic database.

Limitations

• Results on several tasks remain below AlphaEvolve or are not directly comparable due to modified evaluators (ThirdAutoCorrIneq typo fix, CirclePacking tolerance).
• Reward-shaping parameters required manual tuning and transferred poorly across models.
• Only two small open models and a limited set of tasks are evaluated.

Experimental setup

Datasets: circle packing evolution tasks (CirclePacking-T, CirclePacking) first, second, third autocorrelation inequality tasks Hadamard matrix task

Benchmarks: CirclePacking-T (n=26) CirclePacking (n=26) FirstAutoCorrIneq SecondAutoCorrIneq ThirdAutoCorrIneq HadamardMatrix (n=29)

Metrics: best objective score mean objective score across three seeds wall-clock time

Baselines: AlphaEvolve OpenEvolve ShinkaEvolve static-environment RL format-reward RL

Builds on: AlphaEvolve OpenEvolve GRPO slime

Closest papers by meaning

• TTRL: Test-Time Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
• T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling (ICML 2025 · similarity 0.89)
• Revolve: Optimizing AI Systems by Tracking Response Evolution in Textual Optimization (ICML 2025 · similarity 0.88)
• Representation-Based Exploration for Language Models: From Test-Time to Post-Training (ICLR 2026 · similarity 0.88)
• EVOLvE: Evaluating and Optimizing LLMs For In-Context Exploration (ICML 2025 · similarity 0.87)
• Reinforcement Learning Finetunes Small Subnetworks in Large Language Models (NeurIPS 2025 · similarity 0.87)