Research map / Large language models / Agents, code and math
Large Language Models as Optimizers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OPRO, which prompts an LLM with a meta-prompt of previously scored solutions to iteratively generate better solutions, and applies it to linear regression, TSP, and prompt optimization for GSM8K and BBH.
Problem
Gradient-free optimization is hard when the objective is only accessible through natural language or black-box evaluation, and prompt optimization is difficult when only API access to the LLM is available.
Approach
Use an LLM as the optimizer by placing the task description, exemplars, and the trajectory of previously generated solutions with their scores into a meta-prompt; the LLM generates new candidates that are evaluated and appended to the trajectory, iterating without gradients or explicit update rules.
Key results
- Best OPRO-optimized GSM8K instruction reaches 80.2% test accuracy with PaLM 2-L scorer, versus 71.8% for 'Let's think step by step.'
- On BBH, optimized instructions outperform 'Let's think step by step.' by over 5% on 19/23 tasks with the PaLM 2-L scorer.
- On TSP with n=10, all LLMs find optimal solutions; with n=50, gpt-4 reaches 11.0% optimality gap versus 9.8% for farthest insertion.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Optimize the instruction iteratively using the OPRO trajectory | Prompt instruction · GSM8K test, PaLM 2-L scorer | accuracy: 71.8 -> 80.2 | Helped |
| Optimize instructions with OPRO | Prompt instruction · BBH tasks | accuracy: over 5% on 19/23 tasks; 15/23 tasks | Helped |
| Use OPRO instead of one-step generation of 50 instructions (ablation) | Prompt optimization · GSM8K | test accuracy: 60.8 -> 76.3 | Helped |
| Use OPRO instead of one-step generation of 50 instructions (ablation) | Prompt optimization · BBH sports_understanding | test accuracy: 80.0 -> 84.5 | Helped |
| Sort prior instructions from lowest to highest score (ablation) | OPRO meta-prompt · GSM8K and BBH sports_understanding | final accuracy and convergence speed | Helped |
| Include instruction accuracy scores in the trajectory (ablation) | OPRO meta-prompt · GSM8K and BBH sports_understanding | optimization performance | Helped |
| Include task exemplars in the meta-prompt (ablation) | OPRO meta-prompt · GSM8K and BBH sports_understanding | optimization performance | Helped |
| Increase exemplars from 3 to 10 (ablation) | OPRO meta-prompt · GSM8K and BBH sports_understanding | optimization performance | No clear effect |
| Generate 8 instructions per optimization step (ablation) | OPRO · GSM8K and BBH sports_understanding | accuracy | Helped |
| Change the initial instruction (ablation) | OPRO with text-bison scorer · Prompt optimization | accuracy | No clear effect |
| Start from “Let's think step by step.” rather than the empty string or “Let's solve the problem.” (ablation) | OPRO with PaLM 2-L scorer · Prompt optimization | accuracy | Helped |
| Set optimizer temperature to 1.0 (ablation) | OPRO · GSM8K and BBH sports_understanding | accuracy | Helped |
Surprising findings
- Semantically similar instructions can yield drastically different accuracies (e.g., 71.8 vs 49.4 on GSM8K).
- Performance degrades sharply on larger TSP instances, where the heuristic farthest insertion outperforms all LLMs.
Limitations
- Context window limits make large-scale problems hard to fit in the meta-prompt
- Optimization on bumpy landscapes (e.g., Rosenbrock) gets stuck at non-optimal points
- Prompt optimization overfits the small training subset, with training accuracy 5-20% higher than test
Experimental setup
Datasets: GSM8K Big-Bench Hard MultiArith AQuA
Benchmarks: GSM8K Big-Bench Hard MultiArith AQuA
Metrics: test accuracy optimality gap number of steps number of unique solutions explored
Baselines: Let's think step by step Let's work this out in a step by step way EvoPrompt (GA) EvoPrompt (DE) Nearest Neighbor Farthest Insertion one-step instruction generation
Builds on: chain-of-thought prompting APE APO EvoPrompt
Closest papers by meaning
- Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizers (AAAI 2025 · similarity 0.91)
- GReaTer: Gradients Over Reasoning Makes Smaller Language Models Strong Prompt Optimizers (ICLR 2025 · similarity 0.90)
- Localized Zeroth-Order Prompt Optimization (NeurIPS 2024 · similarity 0.89)
- metaTextGrad: Automatically optimizing language model optimizers (NeurIPS 2025 · similarity 0.88)
- Query-Dependent Prompt Evaluation and Optimization with Offline Inverse RL (ICLR 2024 · similarity 0.88)
- Scaling Prompt Synthesis for Large Language Model Reasoning (ICML 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.