Research map / Reinforcement learning / Offline RL
Conservative Bayesian Model-Based Value Expansion for Offline Policy Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CBOP instead of previous model-based methods | D4RL MuJoCo benchmark · four behavior-policy configurations, averaged | normalized score: 116.4% over MOPO; 23.2% over MOReL; 23.7% over COMBO | Helped |
| Replace the full posterior LCB target with MAP target estimation (STEVE) (ablation) | CBOP · D4RL MuJoCo tasks | policy performance | Hurt |
| Use a fixed uniform or geometric weighting instead of adaptive posterior weights (ablation) | CBOP · D4RL environments and tasks | policy performance | Mixed |
| Replace Bayesian posterior estimation with a quantile-based conservative estimate (ablation) | CBOP · all tasks considered in the quantile comparison | policy performance | Hurt |
| Estimate each rollout horizon with independently generated samples rather than reusing the forward sampling (ablation) | CBOP · offline policy learning | final performance | No clear effect |
| Use a Student-t posterior likelihood instead of the Gaussian likelihood (ablation) | CBOP · D4RL experiments | policy performance | No clear effect |
| Use a conservative lower confidence bound from the full posterior instead of the posterior mean (ablation) | CBOP · offline policy optimization | value overestimation | Helped |
| Adjust rollout-horizon weights from ensemble uncertainty instead of fixing a single horizon | CBOP · offline policy optimization | expected rollout horizon | Helped |
Closest papers by meaning
- OCEAN-MBRL: Offline Conservative Exploration for Model-Based Offline Reinforcement Learning (AAAI 2024 · similarity 0.92)
- Constrained Latent Action Policies for Model-Based Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.92)
- Bayesian Design Principles for Offline-to-Online Reinforcement Learning (ICML 2024 · similarity 0.91)
- Iteratively Refined Behavior Regularization for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.91)
- Enhancing Value Function Estimation through First-Order State-Action Dynamics in Offline Reinforcement Learning (ICML 2024 · similarity 0.91)
- From Static Policies to Adaptive Priors in Offline Reinforcement Learning (NeurIPS 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.