Research map / Reinforcement learning / Policy gradient and actor-critic
Towards Deeper Deep Reinforcement Learning with Spectral Normalization
Johan Bjorck, Carla P Gomes, Kilian Q Weinberger
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add spectral normalization to modern heads (ablation) | DRQv2 (DDPG) · walker walk | return: 6.10 $\pm$ 9.08 -> 739.94 $\pm$ 20.88 | Improved |
| Replace MLP heads with modern architecture, without spectral normalization (ablation) | DRQv2 (DDPG) · walker walk | return: 477.40 $\pm$ 117.96 -> 6.10 $\pm$ 9.08 | Hurt |
| Add spectral normalization to modern heads (ablation) | DRQv2 (DDPG) · hopper stand | return: 2.95 $\pm$ 0.99 -> 796.60 $\pm$ 85.04 | Improved |
| Replace MLP heads with modern architecture, without spectral normalization (ablation) | DRQv2 (DDPG) · hopper stand | return: 403.82 $\pm$ 123.85 -> 2.95 $\pm$ 0.99 | Hurt |
| Add spectral normalization to modern heads (ablation) | DRQv2 (DDPG) · reacher easy | return: 77.22 $\pm$ 19.95 -> 801.83 $\pm$ 66.31 | Improved |
| Add spectral normalization to modern heads (ablation) | DRQv2 (DDPG) · cup catch | return: 107.46 $\pm$ 21.43 -> 975.32 $\pm$ 1.3 | Improved |
| Add spectral normalization to modern heads (ablation) | DRQv2 (DDPG) · cartpole sparse | return: 11.27 $\pm$ 0.16 -> 1000.00 $\pm$ 0.00 | Improved |
| Use modern architecture without spectral normalization instead of MLP heads (ablation) | DRQ (SAC) · DeepMind Control Suite tasks | return | Mixed |
| Increase MLP hidden layers from two to four (ablation) | DRQ (SAC) · DeepMind Control Suite tasks | return | Mixed |
| Apply spectral normalization to modern networks (ablation) | DRQ (SAC) · DeepMind Control Suite tasks | return | Improved |
| Increase modern network depth while using spectral normalization (ablation) | DRQ (SAC) · cartpole swingup sparse | return | Improved |
| Increase architecture capacity from two-hidden-layer MLP to modern network with SN | DRQ (SAC) · DeepMind Control Suite tasks | return: slightly more than 141 | Mixed |
Closest papers by meaning
- Network Sparsity Unlocks the Scaling Potential of Deep Reinforcement Learning (ICML 2025 · similarity 0.88)
- Reinforcement Learning Finetunes Small Subnetworks in Large Language Models (NeurIPS 2025 · similarity 0.87)
- Hyperspherical Normalization for Scalable Deep Reinforcement Learning (ICML 2025 · similarity 0.87)
- Normalizing Flows are Capable Models for Continuous Control (NeurIPS 2025 · similarity 0.87)
- In value-based deep reinforcement learning, a pruned network is a good network (ICML 2024 · similarity 0.87)
- Stable Gradients for Stable Learning at Scale in Deep Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
