Research map / Large language models / Language, safety and interpretability
Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Draws an analogy between gradient-based optimizers and LLM-based prompt optimization, systematically studies update direction and update method choices, and builds GPO, a prompt optimizer using relevance-retrieved trajectory, generation-based refinement, and cosine-decayed edit distance.
Problem
LLM-based prompt optimizers rely on ad hoc meta-prompt designs without principled guidelines; the paper seeks systematic understanding of what makes them effective.
Approach
Maps momentum and descent direction to trajectory and current-prompt information, and learning rate and descent to edit-distance control and refinement strategy; empirically finds relevance-retrieved trajectory, generation-based refinement and cosine-decay edit constraint work best and combines them into GPO.
Key results
- GPO achieves 35.43% average on BBH lite-setting task prompts (Llama-2-7b-chat task model) versus 30.51 for empty and 29.91 for CoT, and beats PE2 (33.43) and OPRO (33.29).
- GPO reaches 39.14 average on MMLU subset vs 38.07 for PE2 and 35.96 for empty prompt.
- Cosine decay with generation-based refinement gives the best update-method configuration, e.g., 43.75 vs 43.09 for fixed constraint on the lite BBH setting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GPO instead of the strongest listed baseline PE2 | Llama-2-7B-chat prompt optimization · BBH | accuracy: 33.43 -> 35.43 | Helped |
| Use GPO instead of the strongest listed baseline PE2 | Llama-2-7B-chat prompt optimization · MMLU Average | accuracy: 38.07 -> 39.14 | Helped |
| Use relevance-based retrieval of past prompts and performance (ablation) | LLM-based prompt optimizer · BBH lite | exact match: 41.07 -> 42.53 | Helped |
| Add reflection to prompt and performance information (ablation) | LLM-based prompt optimizer · BBH lite | exact match: 41.07 -> 40.34 | Hurt |
| Summarize the optimization trajectory (ablation) | LLM-based prompt optimizer · BBH lite | exact match: 41.07 -> 41.03 | Hurt |
| Retrieve recent trajectory items (ablation) | LLM-based prompt optimizer · BBH lite | exact match: 41.07 -> 41.93 | Helped |
| Retrieve trajectory items based on performance importance (ablation) | LLM-based prompt optimizer · BBH lite | exact match: 41.07 -> 41.84 | Helped |
| Use generation-based rather than editing-based refinement (ablation) | LLM-based prompt optimizer · BBH lite | exact match: 42.95 -> 43.75 | Helped |
| Use cosine decay for the edit-distance constraint rather than no control (ablation) | LLM-based prompt optimizer · BBH lite | exact match: 42.61 -> 43.75 | Helped |
| Add warmup to the cosine-decay edit-distance constraint (ablation) | LLM-based prompt optimizer · BBH lite | exact match: 43.75 -> 41.29 | Hurt |
| Start optimization from an instructive initial prompt rather than the default prompt (ablation) | GPO · BBH lite | accuracy: 31.25 / 43.75 -> 32.64 / 44.40 | Mixed |
| Optimize prompts with GPT-4 rather than GPT-3.5-turbo | GPO prompt optimizer · BBH lite | accuracy | Helped |
Surprising findings
- Reflection (prompt+performance+reflection) leads to lower performance than prompt+performance alone.
- Warmup of the edit-distance constraint does not help prompt optimization, unlike in gradient-based training.
Limitations
- Only analogizes to the most common gradient-based optimizers (e.g., not Newton's method).
- Relies on textual update directions rather than numerical signals.
- Experiments use subsampled datasets due to computational limits.
Experimental setup
Datasets: BIG-Bench Hard GSM8K MMLU WSC WebNLG MultiArith AQUA
Benchmarks: BIG-Bench Hard GSM8K MMLU WSC WebNLG MultiArith AQUA
Metrics: accuracy exact match ROUGE-L
Baselines: SGDM APE APO OPRO PE2 Empty Chain-of-thought
Builds on: OPRO APE APO PE2 SGDM
Closest papers by meaning
- GReaTer: Gradients Over Reasoning Makes Smaller Language Models Strong Prompt Optimizers (ICLR 2025 · similarity 0.92)
- Enhancing Large Language Model Performance with Gradient-Based Parameter Selection (AAAI 2025 · similarity 0.92)
- Large Language Models as Optimizers (ICLR 2024 · similarity 0.91)
- Benchmarking Optimizers for Large Language Model Pretraining (NeurIPS 2026 · similarity 0.91)
- On Speeding Up Language Model Evaluation (ICLR 2025 · similarity 0.90)
- No Loss, No Gain: Gated Refinement and Adaptive Compression for Prompt Optimization (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.