Research map / Large language models / Language, safety and interpretability
Large Language Models are Human-Level Prompt Engineers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Search and select model-generated instructions instead of using the Greedy instruction-generation baseline | InstructGPT on Instruction Induction · 24 Instruction Induction tasks | zero-shot test performance | Helped |
| Search and select model-generated instructions instead of human-engineered prompts | InstructGPT · 24 Instruction Induction tasks | interquartile mean: 0.749 vs 0.810 | Helped |
| Generate and select instructions with APE instead of using human prompts | InstructGPT · BIG-Bench Instruction Induction (21 tasks) | normalized zero-shot performance | Mixed |
| Prepend an APE-generated instruction to few-shot in-context demonstrations | InstructGPT · 24 Instruction Induction tasks | few-shot test performance: 21 of 24 tasks | Mixed |
| Select the instruction using few-shot execution accuracy rather than zero-shot execution accuracy (ablation) | APE few-shot prompting · 24 Instruction Induction tasks | few-shot test performance | Helped |
| Use the APE-discovered CoT trigger instead of “Let's think step by step.” | text-davinci-002 · MultiArith | accuracy: 78.7 to 82.0 | Helped |
| Use the APE-discovered CoT trigger instead of “Let's think step by step.” | text-davinci-002 · GSM8K | accuracy: 40.7 to 43.0 | Helped |
| Add iterative Monte Carlo resampling around high-scoring instructions (ablation) | APE · six Instruction Induction tasks | test execution accuracy | Mixed |
| Increase the number of instruction candidates sampled (ablation) | APE selection · six Instruction Induction tasks | test accuracy: 4 to 128; 64 instruction samples | Helped |
| Use execution accuracy rather than log probability to score candidate instructions (ablation) | APE · 24 Instruction Induction tasks | correlation with test accuracy | Helped |
| Change the instruction-generation meta-prompt template (ablation) | APE · Instruction Induction tasks | test accuracy | Mixed |
| Use APE-generated prompts to steer a model other than the one used for generation and selection (ablation) | APE-generated instructions · cross-model transfer | performance | Hurt |
Closest papers by meaning
- Scaling Prompt Synthesis for Large Language Model Reasoning (ICML 2026 · similarity 0.91)
- Evaluating the Zero-shot Robustness of Instruction-tuned Language Models (ICLR 2024 · similarity 0.91)
- GReaTer: Gradients Over Reasoning Makes Smaller Language Models Strong Prompt Optimizers (ICLR 2025 · similarity 0.90)
- InstructZero: Efficient Instruction Optimization for Black-Box Large Language Models (ICML 2024 · similarity 0.90)
- Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Models (ICLR 2025 · similarity 0.90)
- Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting (ICLR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.