Research map / Multi-agent systems and games / LLM agents

MEMO: Memory-Augmented Model Context Optimization for Robust Multi-Turn Multi-Agent LLM Games

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Agentic benchmarks and orchestration · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MEMO, a weight-free self-play framework that combines tournament-based prompt evolution (TrueSkill selection), a persistent CRUD memory bank of reflected insights, and prioritized replay to optimize inference-time context for multi-turn multi-agent games.

Problem

Multi-turn, multi-agent LLM game evaluations show high run-to-run variance and prompt sensitivity, making win-rate estimates and rankings unreliable; existing prompt optimizers lack persistent memory across rounds.

Approach

Couples retention (a persistent memory bank of structured insights from self-play trajectories, reinjected as priors and used in memory-augmented proposals) with exploration (TrueSkill lower-confidence-bound tournament selection plus inverse-frequency prioritized replay of decisive states), all without updating model weights.

Key results

• MEMO raises mean win rate from 25.1% to 49.5% for GPT-4o-mini and from 20.9% to 44.3% for Qwen2.5-7B-Instruct using 2,000 self-play games per task.
• MEMO reduces mean RSE to 6.4% on GPT-4o-mini versus 43.3% for UnstableBaselines RL and 12.4% for MIPRO.
• Ablation on GPT-4o-mini: tournament-only 27.1%, memory-only 34.2%, tournament+memory 48.1%, full model with replay 50.2% mean win rate.

Surprising findings

• Exploration alone (tournament-style prompt evolution) yields only modest gains; persistent memory is the dominant mechanism.
• Transferred GPT-4o-mini context gives negative transfer on games where stronger target models already perform well (e.g., Grok-4-Fast on KuhnPoker -6.0%, Briscola -8.0%).

Limitations

• Results rely on a fixed opponent pool and a single fixed hyperparameter configuration chosen without per-task tuning.
• RL remains more effective in perfect-information settings, where MEMO underperforms UnstableBaselines on SimpleTak (34.0% vs 47.3%).

Experimental setup

Datasets: TextArena games (SimpleNegotiation, TwoDollar, KuhnPoker, Briscola, SimpleTak) SPIN-Bench

Benchmarks: SimpleNegotiation TwoDollar KuhnPoker Briscola SimpleTak

Metrics: mean win rate relative standard error (RSE) Kendall's tau_b output token cost

Baselines: Default TextArena prompt CoT ToT TextGrad MIPRO GEPA UnstableBaselines SPIRAL

Builds on: TextGrad MIPRO GEPA ExpeL Reflexion

Closest papers by meaning

• From Player to Master: Enhancing Test-Time Learning of LLM Agents via Reinforcement Learning over Memory (ICML 2026 · similarity 0.92)
• MARSHAL: Incentivizing Multi-Agent Reasoning via Self-Play with Strategic LLMs (ICLR 2026 · similarity 0.89)
• $\texttt{Multi}^2$: Hierarchical Multi-Agent Decision-Making with LLM-Based Agents in Interactive Environments (ICML 2026 · similarity 0.88)
• Do LLM Agents Have Regret? A Case Study in Online Learning and Games (ICLR 2025 · similarity 0.88)
• Opponent Shaping in LLM Agents (ICLR 2026 · similarity 0.88)
• Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions (ICLR 2026 · similarity 0.88)