Research map / Multi-agent systems and games / Cooperative multi-agent RL
Situation-Dependent Causal Influence-Based Cooperative Multi-Agent Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SCIC, a MADDPG-based cooperative MARL method that adds an intrinsic reward from situation-dependent causal influence between agents, estimated via interventional uniform actions and a MINE-style conditional mutual information bound with a learned forward dynamics model.
Problem
Cooperative multi-agent RL suffers from weak coordination and inefficient exploration when rewards are sparse or stochastic; existing MI-based intrinsic rewards do not capture when agents can actually influence one another.
Approach
Measure inter-agent causal influence as state-dependent conditional mutual information I(S_{t+1}^j; A_t^i | S_t^i), estimated under interventions with a uniform action policy, and use the mean influence on other agents as an intrinsic reward so agents seek "significant states" where they can control peers.
Key results
- SCIC-MADDPG converges to higher reward than MADDPG on Predator Prey with 3, 4, and 5 predators, with relatively minor advantage at 4 predators.
- SCIC-MADDPG consistently achieves higher reward than PMIC, SI and MADDPG on Cooperative Navigation with 3, 4, and 5 robots.
- On the 5-agent Cooperative Line task SCIC-MADDPG outperforms the other methods, and matches PMIC on the 3-agent case; the ablation shows removing intervention degrades performance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SCIC-MADDPG instead of PMIC, SI, or MADDPG | Cooperative Predator Prey with 3 predators · Cooperative Predator Prey (3 predators) | reward | Helped |
| Use SCIC-MADDPG instead of PMIC, SI, or MADDPG | Cooperative Predator Prey with 4 predators · Cooperative Predator Prey (4 predators) | reward | Helped |
| Use SCIC-MADDPG instead of PMIC, SI, or MADDPG | Cooperative Predator Prey with 5 predators · Cooperative Predator Prey (5 predators) | reward | Helped |
| Use SCIC-MADDPG instead of PMIC, SI, or MADDPG | Cooperative Navigation with 3 robots · Cooperative Navigation (3 robots) | reward | Helped |
| Use SCIC-MADDPG instead of PMIC, SI, or MADDPG | Cooperative Navigation with 4 robots · Cooperative Navigation (4 robots) | reward | Helped |
| Use SCIC-MADDPG instead of PMIC, SI, or MADDPG | Cooperative Navigation with 5 robots · Cooperative Navigation (5 robots) | reward | Helped |
| Use SCIC-MADDPG instead of PMIC, SI, or MADDPG | Cooperative Line Control with 3 agents · Cooperative Line Control (3 agents) | reward | Mixed |
| Use SCIC-MADDPG instead of PMIC, SI, or MADDPG | Cooperative Line Control with 5 agents · Cooperative Line Control (5 agents) | reward | Helped |
| Use uniform-policy intervention sampling instead of sampling actions from the replay buffer (ablation) | SCIC-MADDPG · Cooperative Predator Prey (5 agents) | reward | Helped |
| Vary the intrinsic-reward temperature α across [0, 0.001,0.01,0.1,0.5] (ablation) | SCIC-MADDPG · Cooperative Predator Prey (5 predators) | reward | Helped |
Surprising findings
- Both PMIC and SI obtain lower reward than plain MADDPG in the predator-prey comparison in the reported results, which the authors attribute to behavior coordination being less useful than reaching significant states.
Limitations
- Evaluation limited to MPE-based tasks and MADDPG as the only base algorithm.
- Relies on a learned forward dynamics model and MINE estimator, adding training cost and approximation error.
- Intrinsic reward computed only during centralized training, requiring access to other agents' policies and actions.
Experimental setup
Datasets: Cooperative Predator Prey (MPE) Cooperative Navigation (MPE) Cooperative Line Control (MPE)
Benchmarks: Cooperative Predator Prey Cooperative Navigation Cooperative Line Control
Metrics: episode reward
Baselines: MADDPG PMIC SI (social influence) SCIC w/o Intervention
Builds on: MADDPG MINE social influence PMIC
Closest papers by meaning
- A Principle of Targeted Intervention for Multi-Agent Reinforcement Learning (NeurIPS 2025 · similarity 0.91)
- Causality-Aware Efficient Exploration for Cooperative Multi-Agent Reinforcement Learning (AAAI 2026 · similarity 0.90)
- Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning (ICML 2019 · similarity 0.89)
- Detecting Influence Structures in Multi-Agent Reinforcement Learning (ICML 2024 · similarity 0.87)
- Multi-agent cooperation through learning-aware policy gradients (ICLR 2025 · similarity 0.87)
- Learning Joint Behaviors with Large Variations (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.