Research map / Large language models / Reasoning and chain of thought

AgentTTS: Large Language Model Agent for Test-time Compute-optimal Scaling Strategy in Complex Tasks

NeurIPS 2025 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Studies test-time compute-optimal scaling (model choice and sample budget per subtask) for multi-stage complex tasks and proposes AgentTTS, an LLM-agent that iteratively searches allocations guided by three empirical insights, outperforming HPO baselines in search efficiency.

Problem

Allocating a fixed inference compute budget across the subtasks of a multi-stage LLM pipeline (choosing model size and number of samples per subtask) is combinatorially large and the subtasks' optimal allocations are interdependent, making brute-force search impractical.

Approach

Formulate test-time compute-optimal scaling for multi-stage tasks with a FLOPs-normalized budget unit and repeated sampling with fusion, derive three empirical insights (subtask model preferences, diminishing returns of sampling, upstream-downstream dependence), and encode them into an LLM agent (o3-mini) that generates trials and guidelines from execution feedback in an Agent-Archive-Environment

Key results

• On 2WikiMultiHopQA, AgentTTS reaches 0.72 test EM in 2.5 hours of search versus 0.70 for the best LLM baseline (MLCopilot and LLM_ZS at 12.5 hours; AgentHPO at 8.3 hours).
• On CWQ, AgentTTS reaches 0.78 test EM in 11.1 hours versus 37.4 hours for AgentHPO and 48.4 hours for MLCopilot.
• On TaskBench, AgentTTS reaches 0.53 p-F1, matching MLCopilot and BO, but requires 64.3 hours of search.

Measured effects reported in the paper

Surprising findings

• Under low retrieval quality, smaller models do not surpass even the lowest-performing 70B model, and sampling beyond an optimum yields flat or negative returns.

Limitations

• Experiments use a single model family per subtask and a small training set (50 samples), so generalization to other families or larger training sets is not established.
• Reports no error bars or variance due to computational overhead.
• Search time for some tasks (e.g., TaskBench 64.3 h) remains large.

Experimental setup

Datasets: 2WikiMultiHopQA HotpotQA ComplexWebQuestions WebQSP TaskBench ChatDev

Benchmarks: 2WikiMultiHopQA HotpotQA ComplexWebQuestions WebQSP TaskBench ChatDev

Metrics: exact match retrieval F1 ROUGE-L node F1 parameter F1 consistency (cosine similarity) search time (hours)

Baselines: Random search Bayesian optimization LLM_ZS MLCopilot AgentHPO

Builds on: AgentHPO MLCopilot Bayesian optimization

Closest papers by meaning

• LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling (NeurIPS 2026 · similarity 0.92)
• Every Rollout Counts: Optimal Resource Allocation for Efficient Test-Time Scaling (NeurIPS 2025 · similarity 0.91)
• Provable Scaling Laws for the Test-Time Compute of Large Language Models (NeurIPS 2025 · similarity 0.91)
• Demystifying and Enhancing the Efficiency of Large Language Model Based Search Agents (ICLR 2026 · similarity 0.90)
• Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for LLM Problem-Solving (ICLR 2025 · similarity 0.90)
• Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck (NeurIPS 2026 · similarity 0.90)