Research map / Reinforcement learning / Offline RL
Continuous Doubly Constrained Batch Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CDC's paired policy and value regularizers instead of the unregularized actor-critic baseline (ablation) | Continuous batch RL on D4RL · D4RL 32 tasks | total normalized return: 173.67 -> 1396.99 | Helped |
| Add the policy constraint (reverse-KL log-likelihood penalty) (ablation) | CDC · D4RL 32 tasks | total normalized return: 555.84 -> 1396.99 | Helped |
| Add the extra-overestimation Q-value penalty Δ (ablation) | CDC · D4RL 32 tasks | total normalized return: 299.25 -> 1396.99 | Helped |
| Add Δ regularization to fitted Q evaluation | FQE · D4RL 32 tasks | Pearson correlation of estimated and actual returns: 0.01 -> 0.37 | Helped |
| Use CDC rather than CQL | Batch RL · D4RL 32 tasks | total normalized return: 1245.2 -> 1396.99 | Helped |
| Use CDC rather than RBVE-C | Continuous batch RL · D4RL 32 tasks | total normalized return: 747.21 -> 1396.99 | Helped |
| Use CDC rather than RBVE-A | Continuous batch RL · D4RL 32 tasks | total normalized return: 104.79 -> 1396.99 | Helped |
| Add the extra-overestimation penalty to FQE | FQE · D4RL tasks | tasks where estimated returns correlate negatively with true returns: 15 of 32 -> 4 | Helped |
| Use only the extra-overestimation penalty, without policy regularization (ablation) | CDC · D4RL 32 tasks | total normalized return: 555.84 vs. 1396.99 | Mixed |
| Use only the policy penalty, without the extra-overestimation penalty (ablation) | CDC · D4RL 32 tasks | total normalized return: 299.25 vs. 1396.99 | Mixed |
Closest papers by meaning
- Provably Good Batch Off-Policy Reinforcement Learning Without Great Exploration (NeurIPS 2020 · similarity 0.94)
- A Minimalist Approach to Offline Reinforcement Learning (NeurIPS 2021 · similarity 0.92)
- Off-Policy Deep Reinforcement Learning without Exploration (ICML 2019 · similarity 0.92)
- Policy Regularization with Dataset Constraint for Offline Reinforcement Learning (ICML 2023 · similarity 0.91)
- A Clean Slate for Offline Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
- Constrained Reinforcement Learning Under Model Mismatch (ICML 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.