Research map / Reinforcement learning / Policy gradient and actor-critic
Dropout Q-Functions for Doubly Efficient Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace REDQ's large Q-function ensemble with a small ensemble of dropout Q-functions | REDQ · MuJoCo: Hopper-v2, Walker2d-v2, Ant-v2, Humanoid-v2 | process time per overall loop: 2328 -> 948; 2339 -> 933; 2336 -> 954; 2400 -> 989 | Helped |
| replace REDQ's large Q-function ensemble with a small ensemble of dropout Q-functions | REDQ · MuJoCo: Hopper, Walker2d, Ant, Humanoid | average return | Mixed |
| add dropout and layer normalization to the Q-functions (ablation) | DroQ · Ant and Humanoid | average return | Helped |
| add dropout and layer normalization to the Q-functions (ablation) | DroQ · Ant and Humanoid | estimation bias | Helped |
| use multiple dropout Q-functions instead of one (ablation) | DroQ · MuJoCo environments | average return | Helped |
| add dropout to both target Q-functions and current Q-functions (ablation) | DroQ · Ant and Humanoid | average return | Helped |
| add dropout to target Q-functions (ablation) | DroQ · all MuJoCo environments | estimation bias | Helped |
| add layer normalization to dropout Q-functions (ablation) | DroQ · Ant and Humanoid | average return | Helped |
| replace layer normalization with batch normalization (ablation) | DroQ · Ant and Humanoid | average return | No clear effect |
| replace layer normalization with group normalization while retaining dropout (ablation) | DroQ · Ant and Humanoid | average return | Helped |
| replace layer normalization with group normalization and remove dropout (ablation) | DroQ · Ant and Humanoid | average return | No clear effect |
| remove variance re-scaling from layer normalization (ablation) | DroQ · especially Humanoid | average return | Hurt |
Closest papers by meaning
- ADDQ: Adaptive distributional double Q-learning (ICML 2025 · similarity 0.84)
- Regularized Q-Learning (NeurIPS 2024 · similarity 0.84)
- Entropy-regularized Diffusion Policy with Q-Ensembles for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.84)
- Model-based Offline Reinforcement Learning with Lower Expectile Q-Learning (ICLR 2025 · similarity 0.84)
- Exclusively Penalized Q-learning for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.83)
- Deep Double Q-learning (NeurIPS 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.