Research map / Multi-agent systems and games / Cooperative multi-agent RL

Beyond Training Time, Test-Time Coordination is Essential for Cooperative MARL

NeurIPS 2026 · Multi-agent systems and games › Cooperative multi-agent RL · approach: Cooperative MARL algorithms

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Position paper arguing cooperative MARL should treat test-time coordination as a design axis, formalizing centralized training with mixed execution (CTME) where a coordinator intermittently emits high-level signals, supported by a toy bottleneck game analysis.

Problem

CTDE provides global information only during training and no coordination channel at execution, so teams in centrally monitored long-horizon cooperative systems cannot realign when objectives or conditions drift (the "recoverability gap").

Approach

Introduce CTME, a problem setting in which a coordinator observes global state and intermittently emits high-level signals (sub-goals, skills, roles, latents) while agents act locally; CTDE is a limit case with no signal. Analyzed via correlated equilibrium/CCE welfare gaps and a sequential toy game with coordination period k.

Key results

• Under the symmetric Nash equilibrium, collision probability converges to 1-1/e ≈ 0.632 as team size grows, and per-agent welfare converges to 1/e - c(1-1/e), negative for c > 1/(e-1).
• A coordinator sampling uniform random permutations achieves zero collisions and welfare N; per-agent coordination gain converges to (1+c)(1-1/e).
• In the sequential toy setting, recovery time after task switches scales as Θ(k) (coordination period), not Θ(τ), and welfare loss under coordinator dropout decreases with k.

Surprising findings

• Per-agent coordination gain depends on collision cost c but only weakly on team size N, since competition and lane count effects nearly cancel.

Limitations

• Toy analysis is didactic and does not test deep MARL on realistic benchmarks.
• CTME provides no robustness guarantee against coordinator corruption; a trusted coordinator is assumed.
• Does not make limited-information distributed control tractable (NEXP-complete in general).

Experimental setup

Metrics: social welfare collision probability recovery time after task switch relative welfare loss under coordinator dropout

Baselines: CTDE CTCE communication-based MARL hierarchical MARL

Builds on: CTDE Dec-POMDP correlated equilibrium options framework communication-based MARL

Closest papers by meaning

• Communication-efficient Multi-Agent Reinforcement Learning with Spatiotemporal Information Hub (AAAI 2026 · similarity 0.88)
• Multi-Agent Coordination via Support-Preserving Distillation (NeurIPS 2026 · similarity 0.87)
• Cross-environment Cooperation Enables Zero-shot Multi-agent Coordination (ICML 2025 · similarity 0.86)
• COOP$^2$: Defining, Observing, and Repairing Cooperation in LLM Multi-Agent Systems (NeurIPS 2026 · similarity 0.86)
• Cooperative Policy Agreement: Learning Diverse Policy for Offline MARL (AAAI 2025 · similarity 0.85)
• GTDE: Grouped Training with Decentralized Execution for Multi-agent Actor-Critic (AAAI 2025 · similarity 0.85)