Research map / Multi-agent systems and games / Cooperative multi-agent RL
High-order Interactions Modeling for Interpretable Multi-Agent Q-Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes QCoFr, a value-decomposition MARL method that models arbitrary-order agent interactions with linear-in-agents continued fraction networks plus a variational information bottleneck for credit assignment, evaluated on LBF, SMAC and SMACv2.
Problem
Modeling high-order agent interactions in cooperative MARL without combinatorial explosion while keeping value decomposition interpretable.
Approach
Represents the joint Q-function as a weighted sum of continued fraction ladders (truncated continued fractions equal Padé approximants) whose depth sets the interaction order with O(n) complexity, and computes ladder credits from VIB-derived assistive information plus global state.
Key results
- QCoFr achieves superior performance across almost all SMAC scenarios, especially on super-hard maps 3s5z_vs_3s6z and 6h_vs_8z, compared with nine baselines.
- On SMACv2 (zerg/protoss/terran 5v5), QCoFr is reported to significantly outperform other algorithms across all scenarios.
- Ablation: depth d=2 gives significant gains over d=1 on 2c_vs_64zg and 3s_vs_5z; performance on 6h_vs_8z peaks at d=6 and degrades further.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use QCoFr instead of the compared baselines | MARL value decomposition · SMAC | performance | Helped |
| use QCoFr instead of the compared baselines | MARL value decomposition · SMACv2 | performance | Helped |
| use QCoFr instead of the compared baselines | MARL value decomposition · LBF | performance | No clear effect |
| increase CFN depth from 1 to 2 (ablation) | QCoFr · SMAC 2c_vs_64zg and 3s_vs_5z | performance | Helped |
| increase CFN depth beyond the task-specific optimum (ablation) | QCoFr · SMAC complex coordination maps | performance | Hurt |
| use CFN depth 6 (ablation) | QCoFr · SMAC 6h_vs_8z | performance | Mixed |
| add VIB-based assistive information (ablation) | QCoFr · MARL tasks | performance | Helped |
| replace CFN with CFN-C (ablation) | QCoFr without VIB · SMAC tasks | performance | No clear effect |
| replace CFN with single-feature-only CFN-D (ablation) | QCoFr without VIB · SMAC tasks, especially 6h_vs_8z | performance | Hurt |
Surprising findings
- Deeper CFN (higher interaction order) degrades performance beyond an optimal depth, attributed to overfitting to spurious higher-order correlations.
- CDS fails on LBF, attributed to overemphasis on behavioral diversity without effective coordination.
Limitations
- CFN depth is fixed per task, which may be suboptimal or wasteful.
- LBF tasks are simplified so higher-order advantages are not fully exercised.
- Interpretability analysis is largely qualitative on a single map (5m_vs_6m).
Experimental setup
Datasets: LBF SMAC SMACv2
Benchmarks: LBF SMAC SMACv2
Metrics: win rate interpretability visualizations of agent and coalition contr cosine similarity of agent Q-values
Baselines: VDN QMIX QPLEX CW-QMIX CDS SHAQ GoMARL ReBorn NA^2Q
Builds on: QMIX VDN NA^2Q CoFrNets Padé approximant
Closest papers by meaning
- N$\text{A}^\text{2}$Q: Neural Attention Additive Model for Interpretable Multi-Agent Q-Learning (ICML 2023 · similarity 0.89)
- Towards Understanding Cooperative Multi-Agent Q-Learning with Value Factorization (NeurIPS 2021 · similarity 0.89)
- Beyond Monotonicity: Revisiting Factorization Principles in Multi-Agent Q-Learning (AAAI 2026 · similarity 0.88)
- Exploiting Structure in Offline Multi-Agent RL: The Benefits of Low Interaction Rank (ICLR 2025 · similarity 0.85)
- Long-Horizon Q-Learning: Accurate Value Learning via n-Step Inequalities (NeurIPS 2026 · similarity 0.85)
- Inverse Factorized Soft Q-Learning for Cooperative Multi-agent Imitation Learning (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.