Research map / Multi-agent systems and games / LLM agents

AgentBrew: Offline Tool-Use Agent Learning from Raw Real-World Trajectories

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Agentic benchmarks and orchestration · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AgentBrew, an offline framework that trains tool-use agents from one batch of raw, unfiltered MCP trajectories by retrospectively inferring aligned instructions and weighting each action by PMI-based credit in an SFT objective.

Problem

Training tool-use LLM agents for a specific real-world application when no pre-defined tasks, verifiers, simulators, or large interaction budget are available, and trajectory-level filtering discards most of the data.

Approach

Relabel each trajectory with an instruction inferred from its observed outcomes (retrospective task inference), then decompose the trajectory's PMI with that instruction via the chain rule into per-action credits that weight a sample-weighted SFT loss, so no trajectory is discarded.

Key results

• AgentBrew improves Qwen3-32B by +8.7 Acc / +9.7 Score on average across GitHub, Notion, and PostgreSQL (18.2 Acc / 58.7 Score), surpassing Qwen3-235B (15.9 / 54.3).
• AgentBrew outperforms rejection sampling by +5.9 Acc / +10.3 Score and Vanilla SFT by +7.9 Acc / +10.6 Score on average.
• AgentBrew-14B reaches 20.2 average Acc / 55.3 Score, +11.5 Acc over Qwen3-14B, by reusing data distilled by Qwen3-32B.

Surprising findings

• Rejection sampling with an LLM judge yields 0 Acc on Notion, as the judge fails to reliably assess stateful API task completion.
• A 32B model trained with AgentBrew surpasses the 235B Qwen3 model on average, indicating targeted offline adaptation can beat general scaling.

Limitations

• Evaluated on only three MCP applications.
• Task inference may degrade on extremely long or chaotic trajectories.
• PMI is an informational proxy that may not capture causal action effects in entangled sequences.

Experimental setup

Datasets: mcp-universe mcpmark BIRD Spider

Benchmarks: mcp-universe mcpmark

Metrics: Acc Score

Baselines: TOUCAN Vanilla SFT Rejection Sampling InfoPO GPT-5 Claude Sonnet-4.5 Gemini-3 Pro Qwen3-235B DeepSeek-V3.2-685B GLM-5-744B GPT-OSS-120B AWM-14B

Builds on: hindsight experience replay InfoPO rejection sampling fine-tuning TOUCAN

Closest papers by meaning

• Generalizable End-to-End Tool-Use RL with Synthetic CodeGym (ICLR 2026 · similarity 0.88)
• What Do Agents Learn from Trajectory-SFT: Semantics or Interfaces? (ICML 2026 · similarity 0.87)
• AgentSynth: Scalable Task Generation for Generalist Computer-Use Agents (ICLR 2026 · similarity 0.87)
• Imitation Learning for Multi-turn LM Agents via On-policy Expert Corrections (ICML 2026 · similarity 0.87)
• Learn-by-interact: A Data-Centric Framework For Self-Adaptive Agents in Realistic Environments (ICLR 2025 · similarity 0.87)
• Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Use (ICML 2026 · similarity 0.86)