Research map / Multi-agent systems and games / Cooperative multi-agent RL

Sequential Order Adjustment of Action Decisions for Multi-Agent Transformer (Student Abstract)

AAAI 2025 · Multi-agent systems and games › Cooperative multi-agent RL · approach: Cooperative MARL algorithms · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AOAD-MAT, a Transformer-based multi-agent actor-critic that adds a subtask predicting the next agent to act, so the decision order is learned, trained with a PPO-based loss.

Problem

Existing sequential multi-agent transformer methods (e.g., MAT) fix the order in which agents make decisions and do not explicitly consider whether that order matters for performance in cooperative MARL.

Approach

Adds a next-agent prediction head to the MAT decoder that shares parameters with the action head and dynamically reorders agent action decisions; the ratios of action and next-agent distributions are multiplied in the PPO objective instead of a weighted multitask sum.

Key results

• On SMAC 5m vs 6m, AOAD-MAT reaches 100.0 win rate versus 96.9 for MAT.
• On MA-MuJoCo HalfCheetah (6x1), AOAD-MAT achieves 13686 mean episode reward versus 11970 for MAT, roughly 10% higher median reward per the authors.
• AOAD-MAT matches or ties baselines on MMM2 and 6h vs 8z and is 100.0 on 3s5z vs 3a6z where MAT-adjust drops to 96.9.

Measured effects reported in the paper

Limitations

• Gains on several SMAC scenarios are small or saturated at 100% win rate.
• Evaluated on a single MA-MuJoCo task in the reported results.
• Method description is brief as a student abstract; ablations and implementation details are limited.

Experimental setup

Datasets: StarCraft Multi-Agent Challenge Multi-Agent MuJoCo

Benchmarks: SMAC 5m vs 6m SMAC MMM2 SMAC 6h vs 8z SMAC 3s5z vs 3s6z MA-MuJoCo HalfCheetah (6x1)

Metrics: median win rate mean episode reward

Baselines: MAT MAT-adjust

Builds on: Multi-Agent Transformer PPO HAPPO

Closest papers by meaning

• Sequential Asynchronous Action Coordination in Multi-Agent Systems: A Stackelberg Decision Transformer Approach (ICML 2024 · similarity 0.88)
• Controlling Behavioral Diversity in Multi-Agent Reinforcement Learning (ICML 2024 · similarity 0.87)
• Foundations of Multi-Agent Learning in Dynamic Environments: Where Reinforcement Learning Meets Strategic Decision-Making (AAAI 2025 · similarity 0.86)
• Revisiting Multi-Agent World Modeling from a Diffusion-Inspired Perspective (NeurIPS 2025 · similarity 0.86)
• Offline Multi-Agent Reinforcement Learning via In-Sample Sequential Policy Optimization (AAAI 2025 · similarity 0.86)
• FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning (ICML 2024 · similarity 0.86)