Research map / Multi-agent systems and games / LLM agents
Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LATS, which combines Monte Carlo Tree Search with ReAct-style acting, LM-based value functions (with self-consistency) and self-reflections, using environment feedback to unify reasoning, acting and planning in LMs without training.
Problem
Language models used as agents rely on reflexive, single-trajectory prompting (CoT, ReAct) and cannot plan, explore alternatives, or adapt flexibly to external feedback in decision-making and reasoning tasks.
Approach
Adapt Monte Carlo Tree Search to LM agents by treating LM states as revertible (copy-pasted context), expanding nodes with sampled ReAct actions and environment observations, scoring nodes with an LM value function mixed with self-consistency, and storing LM-generated verbal reflections from failed trajectories as in-context memory.
Key results
- LATS with GPT-4 achieves 92.7% pass@1 on HumanEval, state of the art at the time.
- LATS with GPT-3.5 reaches 75.9 average score and 38.0 success rate on WebShop, above ReAct (53.8) and Reflexion (64.2) and comparable to gradient-based fine-tuning (67.5).
- LATS (ReAct) with GPT-3.5 reaches 0.63 EM on HotPotQA, versus 0.32 for ReAct; LATS (CoT + ReAct) reaches 0.71.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LATS (CoT) instead of RAP | GPT-3.5 · HotPotQA | EM: 0.60 -> 0.62 | Helped |
| Use LATS (CoT + ReAct) instead of Reflexion | GPT-3.5 · HotPotQA | EM: 0.51 -> 0.71 | Helped |
| Increase LATS expansion size from n = 5 to n = 10 (ablation) | LATS (ReAct), GPT-3.5 · HotPotQA | EM: 0.63 -> 0.65 | Helped |
| Use LATS instead of Reflexion | GPT-3.5 · HumanEval | Pass@1: 68.1 -> 83.8 | Helped |
| Use LATS instead of Reflexion | GPT-4 · HumanEval | Pass@1: 91.0 -> 92.7 | Helped |
| Use LATS instead of RAP | GPT-3.5 · MBPP | Pass@1: 71.4 -> 81.1 | Helped |
| Use LATS instead of Reflexion | GPT-3.5 · WebShop | Score: 64.2 -> 75.9 | Helped |
| Use LATS instead of Reflexion | GPT-3.5 · WebShop | SR: 35.0 -> 38.0 | Helped |
| Use LATS instead of RAP | GPT-3.5 · Game of 24 | Success Rate: 0.40 -> 0.44 | Helped |
| Add the LM value heuristic to LATS (ablation) | LATS, GPT-3.5 · HotPotQA | EM: 0.37 -> 0.63 | Helped |
| Use MCTS selection and backpropagation instead of DFS (ablation) | LATS, GPT-3.5 · HotPotQA | EM: 0.42 -> 0.63 | Helped |
| Add self-reflection to LATS (ablation) | LATS (ReAct), GPT-3.5 · HotPotQA | EM: 0.58 -> 0.63 | Helped |
Surprising findings
- Simply combining ToT or RAP search with ReAct acting performs worse than their reasoning-only versions on HotPotQA, indicating decision-making adaptation of search is non-trivial.
- On WebShop, generated reflections are often generic and offer little benefit; ReAct with best-of-k and Reflexion perform similarly.
Limitations
- Higher computational and token cost than ReAct or Reflexion.
- Requires the environment to support reverting to earlier states.
- Benchmarks are relatively simple compared with real-world interactive environments.
Experimental setup
Datasets: HotPotQA HumanEval MBPP WebShop Game of 24
Benchmarks: HotPotQA HumanEval MBPP WebShop Game of 24
Metrics: exact match pass@1 score success rate
Baselines: CoT CoT-SC ReAct Reflexion ToT RAP Fine-tuning IL IL+RL
Builds on: ReAct Reflexion Tree-of-thought Reasoning via planning (RAP) Chain-of-thought Monte Carlo Tree Search
Closest papers by meaning
- Fleet of Agents: Coordinated Problem Solving with Large Language Models (ICML 2025 · similarity 0.92)
- Policy Guided Tree Search for Enhanced LLM Reasoning (ICML 2025 · similarity 0.91)
- Enhancing Decision-Making for LLM Agents via Step-Level Q-Value Models (AAAI 2025 · similarity 0.90)
- Enhancing Decision-Making of Large Language Models via Actor-Critic (ICML 2025 · similarity 0.90)
- ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Search (ICLR 2024 · similarity 0.89)
- A$^2$FM: An Adaptive Agent Foundation Model for Tool-Aware Hybrid Reasoning (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.