Research map / Reinforcement learning / Policy gradient and actor-critic
Asynchronous Methods for Deep Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use asynchronous parallel actor-learners that update a shared model instead of a single online learner | Deep RL methods · Atari | training stability | Helped |
| Use A3C instead of the value-based asynchronous methods | Asynchronous RL methods · Atari | learning performance | Helped |
| Increase actor-learner threads from 1 to 16 (ablation) | A3C · Atari, 7 games | training speedup: 1.0 -> 12.5 | Helped |
| Increase actor-learner threads from 1 to 16 (ablation) | 1-step Q · Atari, 7 games | training speedup: 1.0 -> 24.1 | Helped |
| Increase actor-learner threads from 1 to 16 (ablation) | 1-step SARSA · Atari, 7 games | training speedup: 1.0 -> 22.1 | Helped |
| Increase actor-learner threads from 1 to 16 (ablation) | n-step Q · Atari, 7 games | training speedup: 1.0 -> 17.2 | Helped |
| Share RMSProp gradient statistics across threads instead of maintaining per-thread statistics (ablation) | Asynchronous n-step Q and A3C · Atari | robustness to learning rates and initializations | Helped |
| Use RMSProp with per-thread statistics instead of shared RMSProp (ablation) | Asynchronous n-step Q and A3C · Atari | robustness to learning rates and initializations | Hurt |
| Use momentum SGD instead of shared RMSProp (ablation) | Asynchronous n-step Q and A3C · Atari | robustness to learning rates and initializations | Hurt |
| Add entropy regularization to the policy objective | A3C · Atari and TORCS | exploration | Helped |
| Give parallel value-based threads different exploration rates | Asynchronous Q-learning · Atari | performance | Helped |
| Use n-step returns instead of one-step updates (ablation) | Asynchronous RL algorithms · TORCS, four configurations | learning speed | Helped |
Closest papers by meaning
- Agent-Centric Actor-Critic for Asynchronous Multi-Agent Reinforcement Learning (ICML 2025 · similarity 0.90)
- Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Training (NeurIPS 2025 · similarity 0.87)
- Enabling Realtime Reinforcement Learning at Scale with Staggered Asynchronous Inference (ICLR 2025 · similarity 0.87)
- Finite-Time Global Optimality Convergence in Deep Neural Actor-Critic Methods for Decentralized Multi-Agent Reinforcement Learning (ICML 2025 · similarity 0.86)
- Adaptive Surrogate Gradients for Sequential Reinforcement Learning in Spiking Neural Networks (NeurIPS 2025 · similarity 0.86)
- Representation Learning Enables Scalable Multitask Deep Reinforcement Learning (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.