Research map / Multi-agent systems and games / Cooperative multi-agent RL
Sequential Multi-Agent Dynamic Algorithm Configuration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Seq-MADAC, a sequential multi-agent dynamic algorithm configuration framework with a sequential advantage decomposition network (SADN) that exploits ordered inter-dependencies among hyperparameters, evaluated on Seq-Sigmoid synthetic benchmarks and MOEA/D configuration.
Problem
Dynamic algorithm configuration of multiple heterogeneous hyperparameters that have inherent inter-dependencies (e.g., operator type before operator parameter), which prior MARL-based DAC methods ignore, leading to sub-optimal configurations.
Approach
Formulates DAC as a contextual sequential multi-agent MDP and decomposes the global advantage sequentially across agents (SADN), each agent maintaining an advantage network conditioned on prior agents' actions, with IGM satisfied and a one-step (lambda=0) GAE-style global advantage update.
Key results
- On MOEA/D configuration over DTLZ/WFG problems with 6, 9, 12 dimensions, SADN attains the best average rank (1.75, 2.5, 2.5) among compared methods.
- SADN achieves the best IGD on most problems, e.g., DTLZ2 at 9 dims 3.875e-02 versus 4.057e-02 for ACE.
- On Seq-Sigmoid tasks, SADN outperforms other methods with faster convergence, lower variance and better final performance, while ACE shows clear performance decay on Seq-Sigmoid.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use sequential advantage decomposition to model ordered hyperparameter dependencies | SADN · Seq-Sigmoid and Seq-Sigmoid-Mask | return | Helped |
| Use sequential advantage decomposition to model ordered hyperparameter dependencies | SADN · Seq-Sigmoid-Robust (5/10 dimensions; 1/2 random agents) | return | Helped |
| Use the correct hyperparameter action order instead of the reverse order (ablation) | sequential methods · Seq-Sigmoid variants | return | Helped |
| Use the correct hyperparameter action order instead of the reverse order (ablation) | sequential methods · MOEA/D | IGD | Helped |
| Use ACE's long-chain action-value function updates | ACE · Seq-Sigmoid | return | Hurt |
| Use sequential action information | SADN, ACE, and SAQL · Seq-Sigmoid-Mask | return | Helped |
| Apply SADN to MOEA/D configuration | MOEA/D · DTLZ and WFG problems | IGD | Mixed |
Surprising findings
- ACE shows clear performance decay on Seq-Sigmoid, attributed to its long-chain action-value update being disrupted by diverse instances.
- Reversing the action order generally underperforms the correct order, except for HAPPO, whose ordering concerns agent update order rather than action order.
Limitations
- The hyperparameter order must be provided in advance by experts.
- Cross-dimensional generalization remains challenging, and SADN underperforms some baselines on some testing problems.
Experimental setup
Datasets: Sigmoid Seq-Sigmoid Seq-Sigmoid-Mask Seq-Sigmoid-Robust DTLZ WFG
Benchmarks: Seq-Sigmoid Seq-Sigmoid-Mask Seq-Sigmoid-Robust MOEA/D on DTLZ and WFG
Metrics: return IGD average rank Wilcoxon rank-sum test win/loss counts
Baselines: MOEA/D ACE SAQL VDN QMIX MAPPO HAPPO HASAC MAPPOar
Builds on: ACE SAQL MADAC HAPPO QMIX
Closest papers by meaning
- Controlling Behavioral Diversity in Multi-Agent Reinforcement Learning (ICML 2024 · similarity 0.86)
- Graph-Supported Dynamic Algorithm Configuration for Multi-Objective Combinatorial Optimization (ICML 2025 · similarity 0.85)
- ConfigX: Modular Configuration for Evolutionary Algorithms via Multitask Reinforcement Learning (AAAI 2025 · similarity 0.85)
- Sequential Order Adjustment of Action Decisions for Multi-Agent Transformer (Student Abstract) (AAAI 2025 · similarity 0.85)
- Offline Multi-Agent Reinforcement Learning via In-Sample Sequential Policy Optimization (AAAI 2025 · similarity 0.84)
- Value-Based Deep Multi-Agent Reinforcement Learning with Dynamic Sparse Training (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.