Research map / Multi-agent systems and games / Human-AI and goal-conditioned
VolleyBots: A Testbed for Multi-Drone Volleyball Game Combining Motion Control and Strategic Play
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces VolleyBots, a GPU-parallel Isaac Sim-based multi-drone volleyball testbed with single-, cooperative- and competitive-task curricula, benchmarks RL/MARL/game-theoretic baselines, a simple hierarchical policy (69.5% win rate vs SP in 3v3), and zero-shot sim-to-real Solo Bump.
Problem
Existing robot sports testbeds lack a combination of agile 3D drone control, turn-based interaction, and mixed cooperative-competitive play, so policies cannot be benchmarked on joint motion control and strategic play.
Approach
A drone volleyball environment built on OmniDrones/Isaac Sim with a task curriculum from single-drone drills to 1v1, 3v3 and 6v6 matches, plus a hierarchical policy that uses PPO-trained low-level drills selected by a rule-based high-level strategy triggered at each hit.
Key results
- PPO outperforms DQN, DDPG, TD3 and SAC in all single-agent tasks under both CTBR and PRT (e.g., Solo Bump PRT 10.83 vs TD3 5.29).
- MAPPO/HAPPO/MAT complete cooperative tasks while QMIX and MADDPG fail; reward shaping improves Set and Spike (Easy) from 0.25 to 0.99 for MAPPO.
- The hierarchical policy achieves a 69.5% win rate against the SP baseline in 3 vs 3 over 1,000 episodes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PPO instead of off-policy RL methods | Single-agent tasks · Back and Forth, Hit the Ball, and Solo Bump | task performance | Helped |
| Use PRT instead of CTBR (ablation) | Single-agent tasks · Back and Forth, Hit the Ball, and Solo Bump | task performance | Helped |
| Add reward shaping (ablation) | Cooperative MARL methods · Bump and Pass | bumps: 11.32 \pm 0.91 -> 13.71 \pm 0.58 | Helped |
| Add reward shaping (ablation) | Cooperative MARL methods · Set and Spike (Easy) | success rate: 0.25 \pm 0.00 -> 0.99 \pm 0.00 | Helped |
| Add reward shaping (ablation) | Cooperative MARL methods · Set and Spike (Hard) | success rate: 0.25 \pm 0.00 -> 0.80 \pm 0.11 | Helped |
| Use on-policy MARL methods instead of off-policy methods | Cooperative tasks · Bump and Pass, Set and Spike (Easy), and Set and Spike (Hard) | task completion | Helped |
| Use a hierarchical policy with PPO low-level drills and rule-based high-level decisions | 3 vs 3 · 3 vs 3 | win rate against SP: 69.5\% | Helped |
| Scale the competitive task from 3 vs 3 to 6 vs 6 | Competitive algorithms · 6 vs 6 | effective strategy convergence | Hurt |
Surprising findings
- In the 6 vs 6 competitive task none of the game-theoretic methods converge to an effective strategy, indicating limited scalability.
- DQN fails entirely on continuous-control drone tasks, scoring zero in several single-agent tasks.
Limitations
- Sim-to-real validated only on the Solo Bump task.
- Relies on fully state-based observations without visual input.
- Traditional drone control baselines were not included.
Experimental setup
Benchmarks: VolleyBots single-agent tasks VolleyBots multi-agent cooperative tasks VolleyBots 1 vs 1 and 3 vs 3 competitive tasks
Metrics: number of round trips hitting distance number of bumps success rate exploitability win rate Elo rating
Baselines: DQN DDPG TD3 SAC QMIX MADDPG HAPPO MAT SP FSP PSRO_Uniform PSRO_Nash
Builds on: OmniDrones Isaac Sim PPO MAPPO PSRO
Closest papers by meaning
- Mimicking To Dominate: Imitation Learning Strategies for Success in Multiagent Games (NeurIPS 2024 · similarity 0.85)
- Foundations of Multi-Agent Learning in Dynamic Environments: Where Reinforcement Learning Meets Strategic Decision-Making (AAAI 2025 · similarity 0.84)
- FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning (ICML 2024 · similarity 0.84)
- Transformers as Game Players: Provable In-context Game-playing Capabilities of Pre-trained Models (NeurIPS 2024 · similarity 0.83)
- The CoachAI Badminton Environment: Bridging the Gap between a Reinforcement Learning Environment and Real-World Badminton Games (AAAI 2024 · similarity 0.83)
- Learning Strategy Representation for Imitation Learning in Multi-Agent Games (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.