Research map / Reinforcement learning / Visual and representation RL
Gymnasium: A Standard Interface for Reinforcement Learning Environments
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Presents Gymnasium, the maintained successor to OpenAI Gym, describing its standardized RL environment API (termination/truncation split, vector environments with reset modes, functional FuncEnv API, algebraic spaces) and built-in environments.
Problem
RL research is hindered by a lack of standardization in environment and algorithm implementations, and OpenAI Gym is no longer maintained, making comparison and reuse of RL work difficult.
Approach
A maintained, standardized Env API that separates termination from truncation, supports vectorized environments with three autoreset modes, adds a functional FuncEnv API aligned with POMDP formalism (JAX-vectorizable), and extends spaces with Text, Graph, Sequence and OneOf.
Key results
- Gymnasium has over 18 million installations/downloads from November 2023 to May 2025, with 800+ pull requests from over 40 contributors.
- Vectorization benchmarks show Async vectorization helps on a high-RAM MacBook Pro (128 GB) for Lunar Lander but degrades on weaker Google Colab hardware.
- Gymnasium splits the single OpenAI Gym done flag into separate termination and truncation signals and supports three autoreset modes (next-step, same-step, disabled).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AsyncVectorEnv instead of SyncVectorEnv | LunarLander vectorization on a high-RAM M3 MacBook Pro · LunarLander, 10,000 total environment steps | steps per second | Helped |
| Use AsyncVectorEnv instead of SyncVectorEnv | LunarLander vectorization on Google Colab · LunarLander, 10,000 total environment steps | steps per second | Hurt |
| Use a custom vectorized implementation with tensor operations | CartPole · CartPole | steps per second: an order of magnitude faster | Helped |
Surprising findings
- Async vectorization can be slower than sync on weaker hardware because subprocess overhead dominates for Lunar Lander.
- Few users and training libraries correctly distinguished termination from truncation under the OpenAI Gym API.
Limitations
- Solely an environment API; users must rely on separate training libraries to train agents.
- Implemented in Python, so environments can be slow compared with C/C++/Cython-based environments.
Experimental setup
Metrics: steps per second download counts GitHub contributors and PRs
Baselines: OpenAI Gym dm_env
Builds on: OpenAI Gym POMDP formalism JAX
Closest papers by meaning
- The CoachAI Badminton Environment: A Novel Reinforcement Learning Environment with Realistic Opponents (Student Abstract) (AAAI 2024 · similarity 0.83)
- Reasoning Gym: Reasoning Environments for Reinforcement Learning with Verifiable Rewards (NeurIPS 2025 · similarity 0.82)
- The CoachAI Badminton Environment: Bridging the Gap between a Reinforcement Learning Environment and Real-World Badminton Games (AAAI 2024 · similarity 0.82)
- Robust Gymnasium: A Unified Modular Benchmark for Robust Reinforcement Learning (ICLR 2025 · similarity 0.81)
- SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Cores (ICLR 2024 · similarity 0.81)
- Uni-RLHF: Universal Platform and Benchmark Suite for Reinforcement Learning with Diverse Human Feedback (ICLR 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.