Research map / Multi-agent systems and games / Cooperative multi-agent RL

The Dormant Neuron Phenomenon in Multi-Agent Reinforcement Learning Value Factorization

NeurIPS 2024 · Multi-agent systems and games › Cooperative multi-agent RL · approach: Value factorization · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Identifies a dormant neuron phenomenon in the mixing networks of MARL value factorization methods and proposes ReBorn, which transfers weights from over-active to dormant neurons while preserving learned action preferences.

Problem

Value factorization MARL mixing networks accumulate dormant (inactive) neurons during training, reducing expressivity; existing single-agent dormant-neuron/reset methods may forget cooperative knowledge.

Approach

Dormant neurons correlate with over-active neurons carrying large activation scores; ReBorn periodically copies (scaled) weights from over-active neurons to randomly chosen dormant neurons in the mixing network, and proves this preserves action preference ordering (Knowledge Invariant principle) and IGM for QMIX, QPLEX, DMIX and RMIX.

Key results

• ReBorn improves QMIX, QPLEX, and RMIX test win rates on MMM2 and 3s5z_vs_3s6z while reducing the dormant ratio of the mixing network.
• ReBorn outperforms ReDo, ReSet, SR and MARR when applied to QMIX on MMM2, 27m_vs_30m and predator-prey large.
• Applying ReBorn to the whole network (violating KI) degrades QMIX and QPLEX performance on MMM2, whereas mixing-network-only ReBorn (satisfying KI) improves them.

Measured effects reported in the paper

Surprising findings

• Dormant neurons are concentrated in the mixing network rather than the agent networks, and their proportion grows with the number of agents.
• Parameter-perturbation methods from single-agent RL (ReDo, ReSet) can violate the IGM principle, and applying perturbation to the whole network can hurt performance.

Limitations

• Evaluated only on SMAC, SMACv2 and predator-prey environments with PyMARL-based value factorization methods.
• Theoretical KI guarantees are shown for the mixing network only; ReBorn on the full network violates KI.
• Hyperparameters (dormant threshold, over-active threshold, interval) require tuning.

Experimental setup

Datasets: SMAC SMACv2 predator-prey

Benchmarks: SMAC SMACv2 predator-prey

Metrics: test win rate return dormant percent over-active sum percent

Baselines: QMIX QPLEX DMIX RMIX ReDo ReSet SR MARR

Builds on: QMIX ReDo ReSet QPLEX PyMARL

Closest papers by meaning

• Stay Hungry, Keep Learning: Sustainable Plasticity for Deep Reinforcement Learning (ICML 2025 · similarity 0.87)
• Measure gradients, not activations! Enhancing neuronal activity in deep reinforcement learning (NeurIPS 2025 · similarity 0.87)
• GradPS: Resolving Futile Neurons in Parameter Sharing Network for Multi-Agent Reinforcement Learning (ICML 2025 · similarity 0.86)
• DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimization (ICLR 2024 · similarity 0.86)
• ConcaveQ: Non-monotonic Value Function Factorization via Concave Representations in Deep Multi-Agent Reinforcement Learning (AAAI 2024 · similarity 0.85)
• REValueD: Regularised Ensemble Value-Decomposition for Factorisable Markov Decision Processes (ICLR 2024 · similarity 0.85)