Research map / Multi-agent systems and games / Games and equilibria
Multi Agent Reinforcement Learning for Sequential Satellite Assignment Problems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes REDA, a MARL method that learns per-agent Q-values for assignments, bootstraps from a greedy assignment solver, and selects joint assignments via a linear assignment mechanism, outperforming IQL, IPPO, and HAAL on satellite constellation assignment.
Problem
Sequential assignment problems in which agents must be assigned to tasks repeatedly over time with state-dependent utilities, where naive MARL leads to selfish behavior and conflicting duplicate assignments.
Approach
Each agent learns Q-values for its possible task assignments (decomposed from the joint Q-function), and joint actions are chosen by solving a linear assignment problem over these learned values, guaranteeing valid and socially optimal assignments; training bootstraps from a greedy solver and uses noise-scaled exploration.
Key results
- REDA drives the group to the optimal joint policy in the 3-agent dictator environment, while IQL and IPPO converge to the greedy joint policy and COMA to (1,3,1).
- In the satellite environment REDA outperforms IQL and IPPO on all metrics, eliminating conflicting assignments while managing power over the full episode, whereas HAAL runs out of power early.
- The abstract reports that REDA outperforms other methods by 20-50% at scale with hundreds of agents and tasks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use REDA with the distributed optimal assignment mechanism instead of independent IQL | Sequential assignment problem · Three-agent, three-task toy SAP | total reward | Helped |
| Use REDA with the distributed optimal assignment mechanism instead of independent IPPO | Sequential assignment problem · Three-agent, three-task toy SAP | total reward | Helped |
| Use REDA instead of COMA | Sequential assignment problem · Three-agent, three-task toy SAP | total reward | Helped |
| Use REDA instead of IQL | Satellite assignment · Satellite constellation environment | percentage of satellites without charge at episode end | Helped |
| Use REDA instead of IPPO | Satellite assignment · Satellite constellation environment | percentage of satellites without charge at episode end | Helped |
| Use REDA instead of IQL | Satellite assignment · Satellite constellation environment | percentage of satellites completing the same task as another satellite | Helped |
| Use REDA instead of IPPO | Satellite assignment · Satellite constellation environment | percentage of satellites completing the same task as another satellite | Helped |
| Use REDA instead of IQL | Satellite assignment · Satellite constellation environment | average number of time steps satellites are assigned to the same tasks | Helped |
| Use REDA instead of IPPO | Satellite assignment · Satellite constellation environment | average number of time steps satellites are assigned to the same tasks | Helped |
| Use REDA instead of HAAL | Satellite assignment · Satellite constellation environment | percentage of satellites without charge at episode end | Helped |
| Use HAAL instead of REDA | Satellite assignment · Satellite constellation environment | average number of time steps satellites are assigned to the same tasks | Helped |
Surprising findings
- Independent agents (IQL, IPPO) often have many satellites simultaneously assigning themselves to the most valuable tasks, even when a task can only be completed by one agent.
- Behavior cloning on the greedy policy was not beneficial for IPPO, so IPPO was trained from scratch.
Limitations
- Assumes reward decomposes additively across agents, which fails with effects such as inter-beam frequency interference.
- Applies only when each agent completes a single task and each task is completed by a single agent.
- Memory-intensive replay buffer (about 300 GB) for the large constellation environment.
Experimental setup
Datasets: simulated satellite constellation (324 satellites, 450 tasks)
Benchmarks: simple 3-agent 3-task dictator environment satellite constellation assignment environment
Metrics: percentage of satellites without charge at end of episode percentage of satellites with duplicate task assignments average time steps satellites share the same task joint reward
Baselines: IQL IPPO COMA HAAL greedy policy
Builds on: DQN EPyMARL COMA IQL IPPO
Closest papers by meaning
- Credit Assignment For Collective Multiagent RL With Global Rewards (NeurIPS 2018 · similarity 0.85)
- PARCO: Parallel AutoRegressive Models for Multi-Agent Combinatorial Optimization (NeurIPS 2025 · similarity 0.84)
- LNS2+RL: Combining Multi-agent Reinforcement Learning with Large Neighborhood Search in Multi-agent Path Finding (AAAI 2025 · similarity 0.84)
- Dr. MAS: Stable Reinforcement Learning for Multi-Agent LLM Systems (NeurIPS 2026 · similarity 0.84)
- OrbitZoo: Real Orbital Systems Challenges for Reinforcement Learning (NeurIPS 2025 · similarity 0.84)
- Scalable Constrained Policy Optimization for Safe Multi-agent Reinforcement Learning (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.