Research map / Large language models / Reasoning and chain of thought
No Loss, No Gain: Gated Refinement and Adaptive Compression for Prompt Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GRACE, a prompt optimization framework that pairs gated refinement (feedback regulation and update rejection gates) with adaptive compression triggered on stagnation, reaching better prompts with under 80 generated prompts.
Problem
Automatic prompt optimization for closed-source LLMs produces unstable updates, is inefficient (hundreds of candidates), and gets trapped in local optima after few steps.
Approach
Deliberately introduce information loss in two places: a feedback regulation gate balances failure signals with success samples when generating a candidate, an update rejection gate accepts only validation-improving candidates, and when K consecutive rejections occur, an adaptive compression step distills the prompt into general core concepts to escape local optima.
Key results
- On BBH with DeepSeek-V3-0324, GRACE averages 94.13 vs 89.42 for PromptAgent (the strongest baseline).
- On domain-specific tasks GRACE averages 82.00 vs 77.60 for PromptAgent; on general NLP tasks 96.63 vs 93.94 for APO.
- GRACE reaches superior final performance using fewer than 80 generated prompts versus roughly 300 for baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GRACE instead of the strongest optimization baseline | BBH · BBH Avg. | accuracy: 89.42 -> 94.13 | Helped |
| Use GRACE instead of the strongest optimization baseline | Domain-specific tasks · Domain-specific Avg. | accuracy: 77.60 -> 82.00 | Helped |
| Use GRACE instead of the strongest optimization baseline | General NLP tasks · General NLP Avg. | accuracy: 93.94 -> 96.63 | Helped |
| Use prompts optimized by GRACE rather than PromptAgent | Reddit TIFU summarization · Reddit TIFU | ROUGE-L: 16.29 -> 17.60 | Helped |
| Use prompts optimized by GRACE rather than PromptAgent | Amazon Fine Food Reviews summarization · Amazon Fine Food Reviews | ROUGE-L: 21.29 -> 23.76 | Helped |
| Increase the share of success samples from 1 of 6 toward a balanced ratio (ablation) | GRACE feedback regulation on TREC · TREC | performance | Mixed |
| Use an excessive success-sample share (5 of 6) instead of a balanced ratio (ablation) | GRACE feedback regulation on TREC · TREC | performance | Hurt |
| Accept every candidate update instead of rejecting non-improving updates (ablation) | GRACE update rejection gate on TREC · TREC | performance | Hurt |
| Remove adaptive compression (ablation) | GRACE on TREC · TREC | performance | Hurt |
| Compress the prompt after optimization stagnates (ablation) | GRACE on TREC · TREC | performance | Helped |
| Vary the success/failure sample ratio while keeping batch size 6 (ablation) | GRACE feedback regulation on TREC · TREC | performance | Mixed |
| Increase optimization steps beyond T=80 (ablation) | GRACE · BBH tasks | performance | No clear effect |
Surprising findings
- Greedy acceptance of every candidate (no update rejection gate) leads to repeated convergence to suboptimal local optima, showing that accepting all updates hurts.
- Prompts optimized on DeepSeek-V3 can underperform the initial prompt when transferred to other models (e.g., Epistemic on GPT-4.1).
Limitations
- Transfer gains of optimized prompts are smaller on models other than the optimization model.
- Hyperparameters (sample ratio, T, K) were selected on the studied tasks and may need tuning on new tasks.
Experimental setup
Datasets: BIG-Bench Hard (Geometric Shapes, Salient Translation, Snarks, Movie Recommendation, Epistemic) NCBI Biosses MedQA TREC Subj CB Reddit TIFU Amazon Fine Food Reviews
Benchmarks: BIG-Bench Hard (5 tasks) NCBI Biosses MedQA TREC Subj CB Reddit TIFU Amazon Fine Food Reviews
Metrics: accuracy F1 ROUGE-L
Baselines: Task (ZS) Task (FS) CoT (ZS) CoT (FS) EvoPrompt OPRO APO PromptAgent
Builds on: APO OPRO EvoPrompt PromptAgent
Closest papers by meaning
- Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizers (AAAI 2025 · similarity 0.90)
- GReaTer: Gradients Over Reasoning Makes Smaller Language Models Strong Prompt Optimizers (ICLR 2025 · similarity 0.89)
- Soft Prompt Recovers Compressed LLMs, Transferably (ICML 2024 · similarity 0.89)
- Fundamental Limits of Prompt Compression: A Rate-Distortion Framework for Black-Box Language Models (NeurIPS 2024 · similarity 0.89)
- Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting (ICLR 2024 · similarity 0.88)
- ADePT: Adaptive Decomposed Prompt Tuning for Parameter-Efficient Fine-tuning (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.