Research map / Architectures and efficiency / Pruning and efficient tuning
The State of Sparse Training in Deep Reinforcement Learning
Laura Graesser, Utku Evci, Erich Elsen, Pablo Samuel Castro
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use sparse training rather than a parameter-equivalent dense network | DQN, SAC, and PPO agents · Atari and MuJoCo | reward | Improved |
| Train sparse networks with pruning instead of dense training | DQN, SAC, and PPO agents · Atari and MuJoCo | reward | Improved |
| Use dynamic sparse training instead of static sparse training | DQN, SAC, and PPO agents · Atari and MuJoCo | reward | Improved |
| Use RigL instead of SET | DQN, SAC, and PPO agents · Atari and MuJoCo | reward | No clear effect |
| Use sparse networks at 80-90% sparsity | DQN and SAC agents · Atari and MuJoCo | reward | No clear effect |
| Use a ResNet instead of the standard CNN | DQN · Atari | IQM score: about 3x improvement in IQM scores | Improved |
| Allocate more parameters to the critic and fewer to the actor (ablation) | SAC · MuJoCo | return | Improved |
| Use ERK instead of uniform sparsity distribution (ablation) | Sparse DQN, SAC, and PPO networks · Atari and MuJoCo | reward | Improved |
| Add a small amount of weight decay (ablation) | Pruning, RigL, and SET · MuJoCo | reward | Improved |
| Use sparsity-aware initialization with uniform layer sparsity (ablation) | Static, RigL, and SET · MuJoCo | reward | Improved |
| Use sparsity-aware initialization with ERK sparsity (ablation) | RigL and SET · MuJoCo | reward | No clear effect |
| Use sparsity-aware initialization with ERK sparsity (ablation) | Static · MuJoCo | reward | Hurt |
Closest papers by meaning
- Network Sparsity Unlocks the Scaling Potential of Deep Reinforcement Learning (ICML 2025 · similarity 0.90)
- Value-Based Deep Multi-Agent Reinforcement Learning with Dynamic Sparse Training (NeurIPS 2024 · similarity 0.88)
- In value-based deep reinforcement learning, a pruned network is a good network (ICML 2024 · similarity 0.87)
- DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimization (ICLR 2024 · similarity 0.87)
- Random Sparse Lifts: Construction, Analysis and Convergence of finite sparse networks (ICLR 2024 · similarity 0.87)
- Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
