Research map / Reinforcement learning / Policy gradient and actor-critic
Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ACKTR instead of A2C | Atari policies · Beamrider, Breakout, Pong, Q-bert, Seaquest, Space Invaders | episode reward: Beamrider: 8148.1 -> 13581.4; Breakout: 581.6 -> 735.7; Pong: 19.9 -> 20.9; Q-bert: 15967.4 -> 21500.3; Seaquest: 1754.0 | Helped |
| Use ACKTR instead of A2C or TRPO | MuJoCo policies · HalfCheetah, Hopper, InvertedPendulum, Reacher | episode reward: HalfCheetah: ACKTR 5586.3, A2C 5343.7, TRPO 5704.7; Hopper: ACKTR 3915.9, A2C 3915.3, TRPO 3755.0; InvertedPendulum: 100 | Mixed |
| Use ACKTR instead of A2C or TRPO | MuJoCo policies · Ant, InvertedDoublePendulum, Swimmer, Walker2d | episode reward: Ant: ACKTR 4621.6, A2C 4870.5, TRPO 5095.0; InvertedDoublePendulum: ACKTR 9356.0, A2C 9356.1, TRPO 9320.0; Swimmer: ACKT | Mixed |
| Use ACKTR instead of A2C | Policies trained from pixels · Reacher, HalfCheetah, Walker2d | final episode reward: 1.6, 2.8, and 1.7 times greater final reward, respectively | Helped |
| Use ACKTR instead of A2C | Atari training · Atlantis | timesteps to reach rewards of 2 million: 2.5 million timesteps vs 25 million timesteps | Helped |
| Apply the Gauss-Newton norm to critic optimization (ablation) | ACKTR actor-critic · HalfCheetah and Breakout | episode reward | Helped |
| Use the Gauss-Newton norm rather than the Euclidean norm for the critic (ablation) | ACKTR actor-critic · HalfCheetah and Breakout | training stability | Helped |
| Estimate critic variance adaptively from Bellman error instead of setting it to 1 (ablation) | ACKTR critic · Atari and MuJoCo experiments | performance | No clear effect |
| Increase batch size from 160 to 640 (ablation) | ACKTR · Breakout | sample efficiency | No clear effect |
| Increase batch size from 160 to 640 (ablation) | A2C · Breakout | sample efficiency | Hurt |
Closest papers by meaning
- Differentiable Trust Region Layers for Deep Reinforcement Learning (ICLR 2021 · similarity 0.88)
- An Implicit Trust Region Approach to Behavior Regularized Offline Reinforcement Learning (AAAI 2024 · similarity 0.86)
- Finite-Time Global Optimality Convergence in Deep Neural Actor-Critic Methods for Decentralized Multi-Agent Reinforcement Learning (ICML 2025 · similarity 0.84)
- Absolute Policy Optimization: Enhancing Lower Probability Bound of Performance with High Confidence (ICML 2024 · similarity 0.84)
- Non-Asymptotic Analysis for Single-Loop (Natural) Actor-Critic with Compatible Function Approximation (ICML 2024 · similarity 0.84)
- Diffusion Policies Creating a Trust Region for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.