Research map / Large language models / Language, safety and interpretability

GReaTer: Gradients Over Reasoning Makes Smaller Language Models Strong Prompt Optimizers

ICLR 2025 · Large language models › Language, safety and interpretability · approach: LLM unlearning and benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes GReaTer, a gradient-over-reasoning prompt optimizer that uses task-loss gradients through generated reasoning to optimize prompts for small open LLMs without large proprietary optimizers.

Problem

Automated prompt optimization for smaller open-source LLMs typically relies on textual feedback from large, expensive closed-source models, and text-space methods ignore finer-grained signals like gradients; reasoning tasks lack fixed verbalizers for standard prompt tuning.

Approach

Compute the cross-entropy loss of answer logits extracted after the model's own generated reasoning (via an extractor prompt), take gradients w.r.t. one-hot indicators over LM-proposed top-k token candidates at each prompt position, pick the top-3 by negative gradient, and keep the candidate with the lowest loss on the training set.

Key results

• On Llama-3-8B, GReaTer reaches 82.6 GSM8K, 68.7 BBH and 62.6 FOLIO, versus 81.1/62.7/58.6 for the best-performing baseline APO on GSM8K and BBH and 62.6 for PE2 on FOLIO (ties).
• On five BBH tasks with Llama-3-8B, GReaTer averages 71.6 versus 62.8 for APE (GPT-4) and 61.0 for OPRO (PaLM-2-L).
• Gradient-over-reasoning ablation shows substantial drops when reasoning is omitted; GReaTer prompts transfer from Llama-3-8B to Gemma-2-9B with BBH-5 average 76.2 vs 69.6 for PE2.

Measured effects reported in the paper

Surprising findings

• Prompts optimized on a small model (Llama-3-8B) transfer to Gemma-2-27B and retain gains over baselines, though the gap narrows as target model size grows.
• Gradient-guided discrete search with the GReaTer objective outperforms AutoPrompt and PEZ on tracking_shuffled_objects_five_objects and movie_recommendation.

Limitations

• Gains are smaller for stronger models such as Gemma-2-9B, where good prompts only offset weaker capability.
• Generated prompts can contain grammatical issues or an informal tone.
• Evaluation covers only 21 selected BBH tasks and two small models, and relies on small train/dev splits.

Experimental setup

Datasets: GSM8K BBH FOLIO

Benchmarks: GSM8K BBH FOLIO

Metrics: accuracy

Baselines: ZS-CoT APE APO PE2 TextGrad OPRO EvoPrompt AutoPrompt PEZ

Builds on: TextGrad AutoPrompt PEZ APO APE

Closest papers by meaning

• Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizers (AAAI 2025 · similarity 0.92)
• Scaling Prompt Synthesis for Large Language Model Reasoning (ICML 2026 · similarity 0.92)
• Large Language Models as Optimizers (ICLR 2024 · similarity 0.90)
• StrategyLLM: Large Language Models as Strategy Generators, Executors, Optimizers, and Evaluators for Problem Solving (NeurIPS 2024 · similarity 0.89)
• No Loss, No Gain: Gated Refinement and Adaptive Compression for Prompt Optimization (NeurIPS 2025 · similarity 0.89)
• Query-Dependent Prompt Evaluation and Optimization with Offline Inverse RL (ICLR 2024 · similarity 0.89)