Research map / Reinforcement learning / Policy gradient and actor-critic
Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BEAR-QL instead of BCQ on medium-quality offline data | Off-policy learning on MuJoCo tasks · Medium-quality data | average evaluation return | Helped |
| Use BEAR-QL instead of TD3 on medium-quality offline data | Off-policy learning on MuJoCo tasks · Medium-quality data | average evaluation return | Helped |
| Use BEAR-QL on random-policy data | Offline learning on continuous-control tasks · Random-policy data | evaluation return | Helped |
| Use BEAR-QL on expert-policy data | Offline learning on continuous-control tasks · Optimal-policy data | evaluation return | Helped |
| Use BEAR-QL instead of BCQ on random-policy data | Offline learning on continuous-control tasks · Random-policy data | evaluation return | Helped |
| Replace the MMD support constraint with a KL-divergence constraint (ablation) | BEAR-QL · Medium-quality data | average return | Hurt |
| Replace the MMD support constraint with a KL-divergence constraint (ablation) | BEAR-QL · HalfCheetah-v2, medium-quality data | average return | Mixed |
| Use a KL-divergence constraint instead of MMD (ablation) | BEAR-QL · Hopper-v2 and Walker2d-v2, medium-quality data | average return | Hurt |
| Use an extensively tuned KL constraint instead of MMD (ablation) | BEAR-QL · Hopper-v2 and Walker2d-v2 | average return | Hurt |
| Use an intermediate MMD sample count, around 4 or 5, rather than a larger count (ablation) | BEAR-QL · Hopper-v2 and Walker2d-v2 | average return | Helped |
| Use 4 MMD samples rather than 10 (ablation) | BEAR-QL · Hopper-v2 and Walker2d-v2 | average return | Helped |
| Use average Q-values rather than minimum Q-values for policy improvement (ablation) | BEAR-QL · Walker2d | performance | Helped |
Closest papers by meaning
- Bootstrapping Fitted Q-Evaluation for Off-Policy Inference (ICML 2021 · similarity 0.92)
- Target Networks and Over-parameterization Stabilize Off-policy Bootstrapping with Function Approximation (ICML 2024 · similarity 0.91)
- Chaining Value Functions for Off-Policy Learning (AAAI 2022 · similarity 0.90)
- Long-Horizon Q-Learning: Accurate Value Learning via n-Step Inequalities (NeurIPS 2026 · similarity 0.88)
- Bootstrap Off-policy with World Model (NeurIPS 2025 · similarity 0.88)
- Iteratively Refined Behavior Regularization for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.