Research map / Reinforcement learning / Policy gradient and actor-critic
Small batch deep reinforcement learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| reduce batch size from 32 to 8 | QR-DQN · Atari 2600 full 60-game suite | average performance improvement: 98.25% | Helped |
| reduce batch size from 32 to 8 | QR-DQN · Atari, 20 games | human-normalized performance: 50% | Helped |
| reduce batch size from 32 to 8 | QR-DQN · Atari, 20 games | wall-time speedup: 29% | Helped |
| reduce batch size | Rainbow, QR-DQN, and IQN · Atari, 20 games | human-normalized IQM | Helped |
| reduce batch size | DQN · Atari, 20 games | human-normalized IQM | No clear effect |
| reduce batch size | Data-efficient Rainbow, SPR, and DrQ(ε) · Atari 100k benchmark, 26 games | human-normalized IQM | No clear effect |
| reduce batch size | Data-efficient Rainbow, SPR, and DrQ(ε) · Atari, 30 million frames | human-normalized IQM | Helped |
| train with reduced batch sizes for longer | QR-DQN · Atari, 200 million frames | human-normalized IQM | Helped |
| use a smaller batch size (ablation) | QR-DQN with CNNx4 or ResNet architecture · Atari, 20 games | human-normalized IQM | Helped |
| set target ε to 0 and reduce batch size (ablation) | QR-DQN · Atari hard-exploration games and 20-game aggregate | human-normalized performance | Helped |
| use smaller batches with multi-step returns (ablation) | DQN, Rainbow, and QR-DQN · Atari, 20 games | human-normalized IQM | Helped |
| increase the n-step value while using smaller batches (ablation) | DQN, Rainbow, and QR-DQN · Atari, 20 games | human-normalized IQM | Helped |
Closest papers by meaning
- Scalable Reinforcement Learning via Adaptive Batch Scaling (ICML 2026 · similarity 0.88)
- Deep Policy Gradient Methods Without Batch Updates, Target Networks, or Replay Buffers (NeurIPS 2024 · similarity 0.86)
- CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicity (ICLR 2024 · similarity 0.86)
- Switching the Loss Reduces the Cost in Batch Reinforcement Learning (ICML 2024 · similarity 0.86)
- A Forget-and-Grow Strategy for Deep Reinforcement Learning Scaling in Continuous Control (ICML 2025 · similarity 0.86)
- On Double Descent in Reinforcement Learning with LSTD and Random Features (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.