Research map / Multi-agent systems and games / LLM agents
Richelieu: Self-Evolving LLM-Based Agents for AI Diplomacy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Richelieu is a self-evolving LLM agent for the full-press game of Diplomacy that combines social reasoning, memory-based reflective planning, negotiation with deception checks, and self-play memory accumulation, reporting modest wins over Cicero without human data.
Problem
Building an AI diplomacy agent that negotiates and plans long-horizon in the press Diplomacy game without relying on large-scale human game data, and that can keep improving over time.
Approach
An LLM-based agent with a social-reasoning module (relationship and intention inference), a reflective planner that retrieves similar past experiences, a negotiation pipeline that assesses the truthfulness of others' statements, and a memory that grows through multi-agent self-play, all in a single LLM-driven architecture.
Key results
- Against Cicero in press Diplomacy (3 Richelieu vs 4 Cicero), Richelieu reaches 6.83% win rate vs Cicero's 6.03%, and about 10% higher score per the authors' scoring system.
- Against AutoGPT, Richelieu wins 9.10% vs AutoGPT's 1.63%.
- Ablation: full Richelieu reaches 6.7% win rate versus 0.4% for the vanilla base LLM (GPT-4) without modules.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add modeling others (ablation) | Richelieu · Press Diplomacy vs Cicero (ablation) | Win rate: 0.4% -> 0.7% | Helped |
| Add sub-goals (ablation) | Richelieu · Press Diplomacy vs Cicero (ablation) | Win rate: 0.7% -> 3.3% | Helped |
| Add negotiation pipeline (ablation) | Richelieu · Press Diplomacy vs Cicero (ablation) | Win rate: 3.3% -> 3.8% | Helped |
| Add reflection with memory (ablation) | Richelieu · Press Diplomacy vs Cicero (ablation) | Win rate: 3.8% -> 5.2% | Helped |
| Add self-play (ablation) | Richelieu · Press Diplomacy vs Cicero (ablation) | Win rate: 5.2% -> 6.7% | Helped |
| Use the Richelieu framework with self-play rather than no self-play | Richelieu with GPT-4.0 · Press Diplomacy vs Cicero | Win rate: 1.5% lower than Cicero's to about 0.7% higher than Cicero's | Helped |
| Use the Richelieu framework with self-play rather than no self-play | Richelieu with Llama 3 · Press Diplomacy vs Cicero | Win rate: 2.3% lower than Cicero's to almost equal to Cicero's | Helped |
| Use the Richelieu framework with self-play rather than no self-play | Richelieu with Spark Desk · Press Diplomacy vs Cicero | Win rate: 3% lower than Cicero's to 0.7% lower than Cicero's | Helped |
| Use the Richelieu framework with self-play rather than no self-play | Richelieu with ERNIE Bot · Press Diplomacy vs Cicero | Win rate: 4% lower than Cicero's to 1.6% lower than Cicero's | Helped |
Surprising findings
- The win-rate gain over Cicero is small in absolute terms (about 0.7 points), with the authors attributing this to draws when several Richelieu agents play together.
Limitations
- Diplomacy action space is constrained and all information is public except negotiation messages, unlike real-world diplomacy.
- Self-play gains plateau and selecting relevant experiences from a growing memory becomes harder.
- Win-rate improvement over Cicero is modest and not statistically characterized.
Experimental setup
Benchmarks: Diplomacy (no-press, massive play) Diplomacy (press, vs Cicero) Diplomacy (press, vs AutoGPT) werewolf game
Metrics: win rate most SC rate survived rate defeated rate C-Diplo score social reasoning success rate
Baselines: Cicero SL-DipNet RL-DipNet BRPI SearchBot DORA AutoGPT
Builds on: Cicero Reflexion AutoGPT
Closest papers by meaning
- Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation (NeurIPS 2024 · similarity 0.85)
- DipLLM: Fine-Tuning LLM for Strategic Decision-making in Diplomacy (ICML 2025 · similarity 0.85)
- Evaluating Generalization Capabilities of LLM-Based Agents in Mixed-Motive Scenarios Using Concordia (NeurIPS 2025 · similarity 0.84)
- Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents (NeurIPS 2024 · similarity 0.84)
- Shapley-Coop: Credit Assignment for Emergent Cooperation in Self-Interested LLM Agents (NeurIPS 2025 · similarity 0.84)
- CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agents (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.