Research map / Multi-agent systems and games / Cooperative multi-agent RL
Sable: a Performant, Efficient and Scalable Sequence Model for MARL
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Sable, a RetNet-based centralised sequence model for online MARL that adapts retention with resettable episode-aware decay, achieving strong performance and linear memory scaling to over a thousand agents.
Problem
Existing MARL methods trade off performance, memory efficiency and scalability; centralised sequence models like MAT are memory-hungry with attention and cannot condition on observation histories.
Approach
Replace MAT's attention with an RL-adapted retention mechanism (encoder-decoder RetNet with cross-retention, agent-block decay, and episode-reset hidden states) trained with the PPO objective under heterogeneous-agent mirror learning, enabling chunkwise training and recurrent constant-memory execution.
Key results
- Sable exceeds baseline performance on 34 of 45 tasks across six environment suites.
- Sable maintains stable performance and linear memory growth on Neom with up to 1024 agents, where MAT runs out of 80GB GPU memory.
- Chunking rollouts up to 16x smaller than full rollout length keeps Sable's performance consistent while reducing memory.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace MAT's attention with Sable's retention and condition on histories | Sable versus MAT · RWARE | episode return: 0.69 -> 0.81 | Helped |
| Use Sable instead of the strongest listed baseline | MARL methods · MABrax | episode return: 0.82 -> 0.56 | Hurt |
| Replace IPPO with Sable | SMAX · SMAX | episode return: 0.93 -> 0.94 | Helped |
| Replace IPPO with Sable | Connector · Connector | episode return: 0.93 -> 0.95 | Helped |
| Replace MAPPO with Sable | LBF · LBF | episode return: 1.0 -> 1.0 | No clear effect |
| Replace MAPPO with Sable | MPE · MPE | episode return: 0.81 -> 0.95 | Helped |
| Add RMSNorm instead of layer normalization (ablation) | MAT · RWARE | performance | Helped |
| Add SwiGLU layers instead of feed forward layers (ablation) | MAT · RWARE | performance | Helped |
| Add RMSNorm and SwiGLU (ablation) | MAT · SMAX | sample efficiency | Helped |
| Add RMSNorm and SwiGLU (ablation) | MAT · SMAX | overall performance | No clear effect |
| Remove temporal memory and condition only on the current timestep (ablation) | Sable · RWARE | performance | Hurt |
| Remove temporal memory and condition only on the current timestep (ablation) | Sable · SMAX | sample efficiency | Hurt |
Surprising findings
- Sable without memory performs similarly to MAT, suggesting the gain comes from temporal memory rather than the retention mechanism itself.
- Off-policy baselines (HASAC, MASAC, QMIX) were tuned with about 15x more gradient updates, and the authors note this would have lowered their reported performance under standard protocols.
Limitations
- Sable underperforms baselines on MABrax, the one environment suite where it does not win on most tasks.
- Scaling-to-many-agents experiments require encoder approximations that drop full self-retention across agents.
- Large-scale scalability is evaluated mainly on the authors' own Neom environment and modified LBF.
Experimental setup
Datasets: RWARE MABrax SMAX Connector LBF MPE Neom
Benchmarks: RWARE MABrax SMAX Connector LBF MPE Neom
Metrics: episode return inter-quartile mean probability of improvement memory usage
Baselines: MAT MAPPO IPPO MASAC HASAC QMIX
Builds on: Retentive Networks (RetNet) Multi-Agent Transformer (MAT) Heterogeneous-Agent Mirror Learning PPO Mava
Closest papers by meaning
- Dispelling the Mirage of Progress in Offline MARL through Standardised Baselines and Evaluation (NeurIPS 2024 · similarity 0.88)
- HyperMARL: Adaptive Hypernetworks for Multi-Agent RL (NeurIPS 2025 · similarity 0.88)
- EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective (NeurIPS 2026 · similarity 0.87)
- Mem-T: Densifying Rewards for Long-Horizon Memory Agents (ICML 2026 · similarity 0.87)
- Foundations of Multi-Agent Learning in Dynamic Environments: Where Reinforcement Learning Meets Strategic Decision-Making (AAAI 2025 · similarity 0.87)
- AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications (ICML 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.