Research map / Reinforcement learning / Visual and representation RL
OrbitZoo: Real Orbital Systems Challenges for Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces OrbitZoo, a PettingZoo-based multi-agent RL environment built on the Orekit high-fidelity astrodynamics library, validated against Starlink ephemerides and tested on Hohmann, collision avoidance, and GEO constellation tasks.
Problem
RL research for satellite maneuvering often uses custom, simplified orbital environments that are time-consuming to build and validate and do not capture real-world perturbations, multi-agent interactions, or uncertainty.
Approach
A standardized, open-source multi-agent RL environment that wraps a high-fidelity industry-standard propagator (Orekit) with PettingZoo POMDP interfaces, polar thrust actions, uncertainty propagation, realistic perturbations, and 3D visualization, validated against real Starlink ephemerides.
Key results
- Validation against Starlink ephemeris achieves a MAPE of 0.16% compared to real-world data.
- Across 31 Starlink satellites over a 16.6-hour propagation, mean RMSE ranges from 24.14 m (low group) to 1924.90 m (high group).
- In the collision avoidance task, PPO outperformed DQN in reducing probability of collision under realistic perturbations, despite training with simplified dynamics.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use continuous-action PPO instead of DQN | Single-agent collision avoidance maneuver · Collision avoidance under realistic evaluation dynamics | Probability of Collision (PoC) | Helped |
| Use continuous action spaces rather than discrete action spaces | OrbitZoo RL missions · Realistic orbital dynamics | Performance | Helped |
Surprising findings
- Policies trained with simplified Newtonian dynamics generalize to full perturbation models in evaluation, with PPO better than DQN at reducing collision risk.
Limitations
- Some satellites in the Starlink validation show large RMSE (mean 1924.90 m), likely due to limited physical property information.
- Collision-avoidance training uses simplified Newtonian attraction, a low-fidelity model for prediction of uncertainty.
Experimental setup
Datasets: Starlink ephemeris data from Space-Track
Benchmarks: Starlink ephemeris validation (31 satellites) Hohmann transfer collision avoidance maneuver GEO constellation coordination
Metrics: MAPE RMSE probability of collision fuel consumption L2 error
Baselines: DQN PPO ColAvGym Poliastro STK
Builds on: Orekit PettingZoo PPO DDPG DQN
Closest papers by meaning
- Multi Agent Reinforcement Learning for Sequential Satellite Assignment Problems (AAAI 2025 · similarity 0.84)
- JaxMARL: Multi-Agent RL Environments and Algorithms in JAX (NeurIPS 2024 · similarity 0.83)
- StraTA: Incentivizing Agentic Reinforcement Learning with Strategic Trajectory Abstraction (NeurIPS 2026 · similarity 0.82)
- Trajectory-Class-Aware Multi-Agent Reinforcement Learning (ICLR 2025 · similarity 0.82)
- Dynamic Optimizations of LLM Ensembles with Two-Stage Reinforcement Learning Agents (ICML 2026 · similarity 0.82)
- OCEAN-MBRL: Offline Conservative Exploration for Model-Based Offline Reinforcement Learning (AAAI 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.