Research map / Multi-agent systems and games / LLM agents
GEM: A Gym for Generalist LLMs
Zichen Liu, Anya Sims, Keyu Duan, Changyu Chen, Simon Yu, Xiangxin Zhou, Haotian Xu, Shaopan Xiong, Bo Liu, Chenmien Tan, Weixun Wang, Hao Zhu and 6 more
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces GEM, an open-source Gym-style environment simulator for training and evaluating LLM agents via experience-based learning.
Problem
LLM training is shifting from static datasets to experience-based learning in interactive environments, but there is no standard environment framework analogous to OpenAI Gym for LLM agents.
Approach
An open-source environment simulator designed for LLMs, analogous to OpenAI-Gym for traditional RL, providing a standardized interface for agent environments to generate experience for training generalist LLMs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Return Batch Normalization to REINFORCE (ablation) | REINFORCE (Qwen3 agents) · 24 GEM environments | episode return | Improved |
| use GRPO instead of REINFORCE/PPO in multi-turn environments with dense rewards | GRPO · GuessTheNumber, Sudoku | episode return | Hurt |
| use GRPO in single-turn ReasoningGym tasks | GRPO · ReasoningGym tasks | episode return | No clear effect |
| use critic-based PPO instead of ReBN | PPO · Sudoku vs Minesweeper | episode return | Mixed |
| lower discount factor gamma (0.999 to 0.9) (ablation) | REINFORCE+ReBN · GuessTheNumber | turns to solve: converges toward optimal about 5.6 turns | Improved |
| add Python tool to RL training (ablation) | Qwen3-4B-Base · math average (5 sets) | Pass@1: 41.4 -> 49.8 | Improved |
| RL training without tool (ablation) | Qwen3-4B-Base · math average | Pass@1: 35.3 -> 41.4 | Improved |
| give base model Python tool without RL (ablation) | Qwen3-4B-Base · MinervaMath | Pass@1: 36.4 -> 30.1 | Hurt |
| add Search tool to RL training (ablation) | Qwen3-4B · QA average (7 sets) | QA score: 23.9 -> 45.5 | Improved |
| train on mixed HotpotQA+NQ instead of HotpotQA only (ablation) | Qwen3-4B with search · QA average | QA score: 43.7 -> 45.5 | Improved |
| use fixed negative reward (-1) instead of ReBN (ablation) | REINFORCE · math:Orz57K / qa:HotpotQA | tool use | Hurt |
| use vanilla REINFORCE with 0/1 reward (no ReBN) (ablation) | REINFORCE · multi-turn tool-use | tool usage | Hurt |
Experimental setup
Builds on: OpenAI Gym
Closest papers by meaning
- ML-Agent: Reinforcing LLM Agents for Autonomous Machine Learning Engineering (ICML 2026 · similarity 0.89)
- Stronger-MAS: Multi-Agent Reinforcement Learning for Collaborative LLMs (ICLR 2026 · similarity 0.89)
- Generalizable End-to-End Tool-Use RL with Synthetic CodeGym (ICLR 2026 · similarity 0.89)
- Group-in-Group Policy Optimization for LLM Agent Training (NeurIPS 2025 · similarity 0.89)
- LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Models (ICML 2025 · similarity 0.88)
- Dr. MAS: Stable Reinforcement Learning for Multi-Agent LLM Systems (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
