Research map / Multi-agent systems and games / Cooperative multi-agent RL

Robust Instruction Compliance in Cooperative Multi-Agent Reinforcement Learning

NeurIPS 2026 · Multi-agent systems and games › Cooperative multi-agent RL · approach: Cooperative MARL algorithms · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MAVIC, a value-correction method that fixes Bellman backup cross-contamination when natural-language instructions interrupt macro-actions in cooperative MARL, evaluated on Box Pushing, Warehouse Tool Delivery and Overcooked.

Problem

Cooperative multi-agent RL with macro-actions must follow natural-language instructions that arrive unpredictably and interrupt ongoing behavior; conditioning rewards on instructions causes Bellman updates to mix value estimates across instruction contexts, degrading compliance or base task performa

Approach

Correct Bellman backups at instruction boundaries by subtracting the value of the incoming instruction and adding back the continuation value under the current instruction, modifying the bootstrapping target (a policy-dependent correction, not potential-based shaping), within an asynchronous macro-action actor-critic with a frozen BERT instruction encoder.

Key results

• On Box Pushing, MAVIC reaches 99% instruction compliance with base return 288.32 vs 290.42 for vanilla Mac-IAC, while the naive baseline reaches only 5% compliance.
• On Overcooked, MAVIC achieves 99% compliance and base return 122.28, whereas the naive baseline drops to 3.62 base return despite 93% compliance.
• On Warehouse Tool Delivery, MAVIC attains 93% compliance and base return 438.90 (std 15.2), compared with 359.12 for naive and 454.49 with 86% compliance for switch training.

Surprising findings

• The naive instruction-reward baseline achieves high compliance in Overcooked yet collapses in base task return because it learns to anticipate instructions that are not active.
• Simply separating data or networks (dual actor-critic) underperforms MAVIC, indicating that value correction at Bellman level, not separation, is needed.

Limitations

• Relies on accurate instruction-conditioned value estimates, which may be high-variance under partial observability and sparse instructions.
• Assumes instruction embeddings generalize across paraphrases; infeasible or far out-of-distribution instructions collapse to a single shared behavior.
• Does not explicitly address exploration over rare or imbalanced instructions.

Experimental setup

Datasets: Box Pushing Warehouse Tool Delivery Overcooked

Benchmarks: Box Pushing Warehouse Tool Delivery Overcooked

Metrics: base task return instruction compliance rate variance across seeds

Baselines: Naive instruction-reward conditioning Switch training Vanilla Mac-IAC Dual-policy Mac-IAC

Builds on: Mac-IAC MacDec-POMDP potential-based reward shaping

Closest papers by meaning

• M³HF: Multi-agent Reinforcement Learning from Multi-phase Human Feedback of Mixed Quality (ICML 2025 · similarity 0.86)
• Empirical Study on Robustness and Resilience in Cooperative Multi-Agent Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
• A Principle of Targeted Intervention for Multi-Agent Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
• HyperMARL: Adaptive Hypernetworks for Multi-Agent RL (NeurIPS 2025 · similarity 0.86)
• Dispelling the Mirage of Progress in Offline MARL through Standardised Baselines and Evaluation (NeurIPS 2024 · similarity 0.86)
• Revisiting Cooperative Off-Policy Multi-Agent Reinforcement Learning (ICML 2025 · similarity 0.86)