Research map / Large language models / Language, safety and interpretability
Evolving Prompts In-Context: An Open-ended, Self-replicating Perspective
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that pruning random ICL demonstrations into unnatural "gibberish" prompts can match or beat prompt optimization baselines, and proposes PromptQuine, an evolutionary search that finds such pruning strategies using only in-context tokens.
Problem
Prompt design for in-context learning is typically driven by well-crafted instructions and demonstrations, yet effective prompt optimization is costly and unreliable; the paper asks whether pruning tokens from random ICL prompts (including into incoherent text) can improve performance and how to sea
Approach
Frames prompt compression as guided prompt search over fixed-order token subsequences of ICL prompts, and uses a genetic algorithm with regularized evolution (PromptQuine) that mutates prompts by pruning random tokens, selecting by held-out fitness, so the search discovers unnatural but effective prompts without external tokens or attribution guidance.
Key results
- PromptQuine on 1-shot ICL with Llama3-8B-It reaches 77.5% average accuracy across six classification tasks, versus 69.6% for original 1-shot ICL and 75.8% for Promptbreeder, with about 52.9% of prompt tokens retained.
- On AdvBench jailbreaking with Vicuna-7b-v1.5, PromptQuine (ASR-LLM guided) raises attack success rate from 54.7% (2-shot ICL) to 97.5%.
- On Yelp sentiment transfer with Llama3-8B-It (greedy), PromptQuine achieves a joint score of 61.0 versus 54.4 for ICL and 59.1 for Promptbreeder.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Prune a 1-shot ICL prompt with PromptQuine | Meta-Llama-3-8B-Instruct · Classification tasks (average accuracy) | accuracy (%): 69.6 -> 77.5 | Helped |
| Prune a 4-shot ICL prompt with PromptQuine instead of using TAPruning | Meta-Llama-3-8B-Instruct · Classification tasks (average accuracy) | accuracy (%): 77.3 -> 81.3 | Helped |
| Prune the zero-shot chain-of-thought trigger prompt | InstructGPT (text-davinci-002) · MultiArith | accuracy: 81.5 -> 86.7 | Helped |
| Use PromptQuine rather than TAPruning | Yelp sentiment transfer · Yelp sentiment transfer (all reported decoding/model settings) | Joint Score: 30.2 -> 33.3; 54.6 -> 57.9; 59.6 -> 61.0; 68.1 -> 72.1 | Helped |
| Prune a 2-shot ICL prompt with PromptQuine | Vicuna-7b-v1.5 · AdvBench | ASR-EM: 50.4 -> 99.3 | Helped |
| Prune a 2-shot ICL prompt with PromptQuine | Mistral-7B-Instruct-v0.3 · AdvBench | ASR-LLM: 47.2 -> 98.1 | Helped |
| Use steering-vector fitness instead of ASR-EM or ASR-LLM fitness (ablation) | PromptQuine · AdvBench | ASR-LLM: 97.4; 97.5; 94.2 | Hurt |
| Remove label words from the pruned prompt (ablation) | PromptQuine-pruned ICL · Classification tasks | performance | Hurt |
| Remove the entire output, including signal words and label words (ablation) | PromptQuine-pruned ICL · Classification tasks | performance | Hurt |
| Remove the entire output (ablation) | PromptQuine-pruned ICL · SNLI | performance | Helped |
| Prune prompts across varied ICL templates (ablation) | PromptQuine · SNLI | accuracy fluctuation: 60.8 -> 75.1 | Mixed |
| Prune prompts across varied ICL templates (ablation) | PromptQuine · PIQA | accuracy fluctuation: 79.5 -> 82.1 | Mixed |
Surprising findings
- Pruned ICL prompts retain label words and perform well even with random verbalizers, where unpruned ICL is near chance, especially for the 70B model.
- Performance gains from token pruning are highly sensitive to ICL template choice; SNLI accuracy fluctuates by up to 14.3 points across templates.
Limitations
- Fixed-order token subsequence search cannot insert or replace tokens, limiting exploration.
- Results are sensitive to ICL template choice, so pruning is not a universally reliable way to stabilize ICL.
- Generation tasks are slower to search because of imperfect fitness feedback, requiring more samples.
Experimental setup
Datasets: SST-2 Subj AG's News Yelp-5 SNLI Yahoo PIQA Yelp sentiment transfer AdvBench GSM8K MAWPS MultiArith
Benchmarks: SST-2 Subj AG's News Yelp-5 SNLI Yahoo PIQA Yelp sentiment transfer AdvBench GSM8K MAWPS
Metrics: accuracy joint score attack success rate
Baselines: ICL (1-shot) ICL (4-shot) LLMLingua LLMLingua2 RLPrompt EvoPrompt Promptbreeder SAHCPruning
Builds on: Promptbreeder RLPrompt Threshold Accepting regularized evolution LLMLingua
Closest papers by meaning
- Scaling Prompt Synthesis for Large Language Model Reasoning (ICML 2026 · similarity 0.91)
- Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution (ICML 2024 · similarity 0.89)
- GReaTer: Gradients Over Reasoning Makes Smaller Language Models Strong Prompt Optimizers (ICLR 2025 · similarity 0.89)
- Prompt Sketching for Large Language Models (ICML 2024 · similarity 0.89)
- How to Auto-optimize Prompts for Domain Tasks? Adaptive Prompting and Reasoning through Evolutionary Domain Knowledge Adaptation (NeurIPS 2025 · similarity 0.89)
- StrategyLLM: Large Language Models as Strategy Generators, Executors, Optimizers, and Evaluators for Problem Solving (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.