Research map / Reinforcement learning / Offline RL
DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add DR3 to CQL | CQL · Atari 2600 17 games, 1%, 5%, and 10% data | IQM normalized average performance: 43.7 -> 56.9; 78.1 -> 105.7; 59.3 -> 65.8 | Helped |
| Add DR3 to REM | REM · Atari 2600 17 games, 1%, 5%, and 10% data | IQM normalized average performance: 4.0 -> 16.5; 25.9 -> 60.2; 53.3 -> 73.8 | Helped |
| Add DR3 to CQL | CQL · D4RL antmaze-umaze-v2, 2M and 3M steps | performance: 84.00 $\pm$ 2.67 -> 85.33 $\pm$ 4.16; 87.00 $\pm$ 1.73 -> 90.00 $\pm$ 4.00 | Helped |
| Add DR3 to CQL | CQL · D4RL antmaze-umaze-diverse-v2, 2M and 3M steps | performance: 45.67 $\pm$ 8.50 -> 40.67 $\pm$ 11.84; 36.33 $\pm$ 7.09 -> 52.00 $\pm$ 11.26 | Mixed |
| Add DR3 to CQL | CQL · D4RL antmaze-medium-play-v2, 2M and 3M steps | performance: 24.00 $\pm$ 28.16 -> 73.00 $\pm$ 4.00; 16.00 $\pm$ 26.85 -> 71.33 $\pm$ 1.52 | Helped |
| Add DR3 to CQL | CQL · D4RL antmaze-medium-diverse-v2, 2M and 3M steps | performance: 32.67 $\pm$ 9.29 -> 67.00 $\pm$ 2.00; 48.33 $\pm$ 6.11 -> 61.67 $\pm$ 3.21 | Helped |
| Add DR3 to CQL | CQL · D4RL antmaze-large-play-v2, 2M and 3M steps | performance: 3.33 $\pm$ 2.51 -> 28.00 $\pm$ 4.35; 0.33 $\pm$ 0.57 -> 26.33 $\pm$ 11.93 | Helped |
| Add DR3 to CQL | CQL · D4RL antmaze-large-diverse-v2, 2M and 3M steps | performance: 1.33 $\pm$ 2.30 -> 25.67 $\pm$ 0.57; 0.00 $\pm$ 0.00 -> 28.33 $\pm$ 1.52 | Helped |
| Add DR3 to CQL | CQL · D4RL kitchen-mixed, kitchen-partial, and kitchen-complete | performance: 27.67 $\pm$ 12.66 -> 37.00 $\pm$ 11.53; 20.67 $\pm$ 15.57 -> 40.67 $\pm$ 4.04; 28.00 $\pm$ 14.73 -> 38.67 $\pm$ 6.66 | Helped |
| Add DR3 with gradients stopped through next-state features (ablation) | REM · Atari 2600 17 games, 1%, 5%, and 10% data | IQM normalized average performance: 4.0 -> 15.0; 25.9 -> 55.5; 53.3 -> 67.7 | Helped |
| Use stop-gradient DR3 instead of full DR3 (ablation) | REM · Atari 2600 17 games | IQM normalized average performance: 15.0 vs 16.5; 55.5 vs 60.2; 67.7 vs 73.8 | Mixed |
| Use a slower target-network update, N=500 (ablation) | FQI and CQL · Gridworld and Atari feature co-adaptation experiments | feature dot product | Hurt |
Closest papers by meaning
- On Double Descent in Reinforcement Learning with LSTD and Random Features (ICLR 2024 · similarity 0.88)
- Adaptive Regularization of Representation Rank as an Implicit Constraint of Bellman Equation (ICLR 2024 · similarity 0.88)
- Revisiting Data Augmentation in Deep Reinforcement Learning (ICLR 2024 · similarity 0.87)
- Parseval Regularization for Continual Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
- Stop Regressing: Training Value Functions via Classification for Scalable Deep RL (ICML 2024 · similarity 0.86)
- Learning to Trust Bellman Updates: Selective State-Adaptive Regularization for Offline RL (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.