Research map / Multi-agent systems and games / LLM agents

AgentGrad: Intervention-guided Prompt Optimization for Multi Agent Systems

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AgentGrad, a textual-gradient prompt optimizer for multi-agent systems that uses sequential hint-injection interventions to find the responsible agent and clusters sample-level gradients into semantic minibatches before abstraction.

Problem

Textual-gradient prompt optimization for LLM multi-agent systems selects target prompts without verifying which agent causes a failure, lacks agent-level supervision, and aggregates unrelated gradients randomly, leading to costly optimization and poor generalization.

Approach

Sequential intervention (reverse-order hint injection into one agent at a time) identifies the target agent whose correction resolves a failure and provides the intervened output as agent-level pseudo-label for gradient extraction; an aggregator LLM clusters sample-level gradients into semantic minibatches and abstracts each cluster into a generalized gradient.

Key results

• With GPT-5-mini, AgentGrad averages +11.76 points over no-optimization baseline and beats GEPA on all five benchmarks (e.g., HotpotQA 73.89 vs 68.33).
• Average wall-clock optimization time is 136 minutes versus 337 for GEPA (2.5x speedup), and average API cost is $27.14 versus $34.73 for GEPA (21.8% reduction).
• With Qwen3-8B, AgentGrad averages +9.67 points over baseline and achieves the best score on HotpotQA, HoVer, PUPA and MATH.

Surprising findings

• On IFBench with Qwen3-8B, AgentGrad (41.42) does not beat TextGrad (42.52), and on PUPA cost AgentGrad is slightly more expensive than MIPROv2 (-3.6%).

Limitations

• Hint injection requires ground-truth outputs or constraints at training time, limiting applicability when labels are unavailable.
• Intervention over agents adds per-failure rollouts; hard cases unresolved after step 1 are excluded from the round.
• Gains on some settings are small or absent (e.g., IFBench with Qwen3-8B, HoVer).

Experimental setup

Datasets: HotpotQA HoVer IFBench PUPA MATH

Benchmarks: HotpotQA HoVer PUPA IFBench MATH 2WikiMultiHopQA EX-FEVER PUPA-TNB IFEval OlympiadBench

Metrics: accuracy/reward score wall-clock time (minutes) API cost (USD) minibatch improvement ratio validation improvement ratio

Baselines: MIPROv2 TextGrad GEPA No prompt optimization baseline

Builds on: TextGrad GEPA MIPROv2 ProTeGi

Closest papers by meaning

• Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies (ICLR 2026 · similarity 0.90)
• MARS: Multi-Agent Adaptive Reasoning with Socratic Guidance for Automated Prompt Optimization (AAAI 2026 · similarity 0.90)
• MASPOB: Bandit-Based Prompt Optimization for Multi-Agent Systems with Graph Neural Networks (ICML 2026 · similarity 0.90)
• MAS$^2$: Self-Generative, Self-Configuring, Self-Rectifying Multi-Agent Systems (ICLR 2026 · similarity 0.89)
• MASPO: Joint Prompt Optimization for LLM-based Multi-Agent Systems (ICML 2026 · similarity 0.89)
• GPTSwarm: Language Agents as Optimizable Graphs (ICML 2024 · similarity 0.89)