Research map / Large language models / Agents, code and math
PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PromptAgent, which frames prompt optimization as MCTS-based strategic planning where an optimizer LLM turns base-model error feedback into prompt edits, producing expert-level prompts across 12 tasks.
Problem
Automatically generating expert-level prompts for API-based LLMs, since existing gradient-free prompt optimizers use local paraphrase-style edits and naive search, missing domain knowledge.
Approach
Treat prompts as MDP states and error-based feedback from the optimizer LLM as actions; search the prompt space with Monte Carlo tree search (UCT selection, expansion with error-driven candidates, simulation, backpropagation) so that domain-rich prompts are found efficiently.
Key results
- PromptAgent averages 0.802 on six BBH tasks vs 0.690 for APE and 0.707 for CoT (few-shot).
- On NCBI, PromptAgent reaches F1 0.645 vs 0.576 for APE and 0.521 for human zero-shot prompt.
- Prompts optimized on GPT-3.5 transfer to GPT-4, beating baselines on 11/12 tasks (average 0.839 vs 0.762 for APE).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| optimize prompts with PromptAgent instead of using CoT prompts | GPT-3.5 · BBH (average) | accuracy: 0.707 -> 0.802 | Helped |
| optimize prompts with PromptAgent instead of APE | GPT-3.5 · Domain-specific tasks (average) | score: 0.582 -> 0.655 | Helped |
| optimize prompts with PromptAgent instead of APE | GPT-3.5 · General NLU tasks (average) | score: 0.778 -> 0.868 | Helped |
| replace MCTS planning with single-step Monte Carlo search (ablation) | PromptAgent · Search ablation tasks (average) | accuracy: 0.754 -> 0.635 | Hurt |
| replace MCTS planning with beam search (ablation) | PromptAgent · Search ablation tasks (average) | accuracy: 0.754 -> 0.697 | Hurt |
| replace MCTS planning with greedy depth-first search (ablation) | PromptAgent · Search ablation tasks (average) | accuracy: 0.754 -> 0.698 | Hurt |
Surprising findings
- Optimized expert prompts transfer poorly to the weaker PaLM 2, where performance drops across most tasks.
- Few-shot CoT beats PromptAgent on Object Counting, the only task where it does not win.
Limitations
- Relies on expensive API calls to GPT-4 as optimizer and GPT-3.5 as base model.
- Expert prompts are long and complex and may not transfer to weaker or smaller models.
- Reward uses a small held-out training subset, so results may vary with its size.
Experimental setup
Datasets: BIG-Bench Hard (Penguins, Geometric Shapes, Epistemic, Object Counting, Temporal Sequences, Causal Judgement) NCBI disease NER Biosses MedQA Subj TREC CB
Benchmarks: BIG-Bench Hard subsets NCBI Biosses MedQA Subj TREC CB
Metrics: accuracy F1
Baselines: Human prompts (ZS) Human prompts (FS) CoT (ZS) CoT GPT Agent APE Monte Carlo search Beam search Greedy search
Builds on: Automatic Prompt Engineer Tree of Thoughts Monte Carlo tree search chain-of-thought
Closest papers by meaning
- StrategyLLM: Large Language Models as Strategy Generators, Executors, Optimizers, and Evaluators for Problem Solving (NeurIPS 2024 · similarity 0.90)
- Scaling Prompt Synthesis for Large Language Model Reasoning (ICML 2026 · similarity 0.90)
- How to Auto-optimize Prompts for Domain Tasks? Adaptive Prompting and Reasoning through Evolutionary Domain Knowledge Adaptation (NeurIPS 2025 · similarity 0.90)
- SePO: Self-Evolving Prompt Agent for System Prompt Optimization (NeurIPS 2026 · similarity 0.89)
- GReaTer: Gradients Over Reasoning Makes Smaller Language Models Strong Prompt Optimizers (ICLR 2025 · similarity 0.88)
- Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.