Research map / Multi-agent systems and games / Cooperative multi-agent RL
A Deeper Understanding of State-Based Critics in Multi-Agent Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a state-based critic instead of a history-based critic | Decentralized history-based policies · Dec-POMDPs with partially observable histories | policy-gradient bias | Hurt |
| Use a state-based critic rather than a history-based critic when its values are unbiased | Policy-gradient estimation · Dec-POMDPs | gradient variance | Hurt |
| Use a state-based critic instead of a history-based critic | Dec-Tiger · Dec-Tiger | Q value: 13.8859 -> -16.175 | Hurt |
| Add history information to the state-based critic | Centralized actor-critic · Dec-Tiger, Box Pushing, Cleaner | training performance | Helped |
| Use a history-based critic rather than a state-based critic | Centralized actor-critic · Dec-Tiger | performance | Helped |
| Use a state-based critic rather than a history-based critic | COMA · Most SMAC scenarios | overall performance | Helped |
| Use any tested critic type | Multi-agent actor-critic · Multi-agent Recycling | learning to recycle large targets | No clear effect |
| Use a reactive policy without a deterministic observation model | State-based critic · Partially observable Dec-POMDPs | policy-gradient bias | Hurt |
| Use a reactive policy together with a deterministic observation model | State-based critic · Partially observable Dec-POMDPs | policy-gradient bias | Helped |
| Add state information to the history-based critic | History-based critic · Dec-Tiger, Box Pushing, Cleaner | performance | Helped |
Closest papers by meaning
- Communication-Efficient Actor-Critic Methods for Homogeneous Markov Games (ICLR 2022 · similarity 0.89)
- Actor-Attention-Critic for Multi-Agent Reinforcement Learning (ICML 2019 · similarity 0.89)
- A Theoretical Justification for Asymmetric Actor-Critic Algorithms (ICML 2025 · similarity 0.88)
- Asynchronous Actor-Critic for Multi-Agent Reinforcement Learning (NeurIPS 2022 · similarity 0.88)
- Towards Principled Unsupervised Multi-Agent Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
- Agent-Centric Actor-Critic for Asynchronous Multi-Agent Reinforcement Learning (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.