Research map / Multi-agent systems and games / Cooperative multi-agent RL

Learning Generalizable Skills from Offline Multi-Task Data for Multi-Agent Cooperation

ICLR 2025 · Multi-agent systems and games › Cooperative multi-agent RL · approach: MARL benchmarks and grouping · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes HiSSD, a hierarchical offline multi-task MARL method that jointly learns common skills (via a planner with forward state prediction and IQL value estimation) and task-specific skills (via a VAE-style controller with contrastive task discrimination) to generalize to unseen SMAC and MAMuJoCo tasks.

Problem

Generalizing cooperative multi-agent policies learned from offline multi-task data to unseen tasks with different numbers of agents and targets, without extra interaction.

Approach

Jointly learns common skills that encode cooperative temporal knowledge via a local-to-global next-state prediction objective with implicit Q-learning weighting, and task-specific skills trained with a contrastive task-discrimination loss to guide fine-grained action execution.

Key results

• On SMAC Marine-Hard expert data, HiSSD reaches 99.2% win rate on unseen 4m and 5m versus 95.3% and 89.1% for ODIS.
• On MAMuJoCo HalfCheetah-v2 unseen tasks, HiSSD scores 3472.12 on back shin (vs. 3217.59 for ODIS) and 4175.29 on front foot (vs. 3930.82 for ODIS).
• Ablation on Marine-Hard target tasks (expert) shows HiSSD at 61.2% vs. 45.8% without the planner (w/o Planning).

Measured effects reported in the paper

Surprising findings

• On large-scale SMAC tasks, the learned task-specific skills overlap across tasks, indicating the method struggles to separate 10m and 12m distributions.
• HiSSD underperforms ODIS on some source and unseen tasks (e.g., 3m medium, 7m8m medium, 13m15m expert), so gains are not uniform.

Limitations

• Training stability is a noted limitation of HiSSD.
• Task-specific skills fail to separate large-scale tasks (10m, 12m) per the visualization.
• Gains are not uniform; several tasks show HiSSD below ODIS or BC baselines.

Experimental setup

Datasets: SMAC offline multi-task datasets (from ODIS) MAMuJoCo HalfCheetah-v2 multi-task dataset

Benchmarks: SMAC Marine-Easy SMAC Marine-Hard SMAC Stalker-Zealot MAMuJoCo HalfCheetah-v2

Metrics: test win rate average return

Baselines: ODIS UPDeT-m BC-t BC-r BC IQL TD3-BC

Builds on: ODIS Implicit Q-learning beta-VAE UPDeT

Closest papers by meaning

• Discovering Generalizable Multi-agent Coordination Skills from Multi-task Offline Data (ICLR 2023 · similarity 0.96)
• Offline Multi-agent Continual Cooperation via Skill Partition and Reuse (ICML 2026 · similarity 0.93)
• Goal-Oriented Skill Abstraction for Offline Multi-Task Reinforcement Learning (ICML 2025 · similarity 0.89)
• Robust Policy Learning via Offline Skill Diffusion (AAAI 2024 · similarity 0.88)
• Cooperative Policy Agreement: Learning Diverse Policy for Offline MARL (AAAI 2025 · similarity 0.88)
• Group of Skills: Group-Structured Skill Retrieval for Agent Skill Libraries (NeurIPS 2026 · similarity 0.88)