Research map / Reinforcement learning / Policy gradient and actor-critic
Conservative Dual Policy Optimization for Efficient Model-Based Reinforcement Learning
Shenao Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use only Referential Update (drop Conservative Update) (ablation) | CDPO · MuJoCo tasks | return | Hurt |
| use only Conservative Update (drop Referential Update) (ablation) | CDPO · MuJoCo tasks | return | Hurt |
| time-decayed trust-region eta instead of constant eta (ablation) | CDPO · MuJoCo tasks | return | No clear effect |
| remove the trust-region constraint (ablation) | CDPO · MuJoCo tasks | return | Hurt |
| swap Dyna solver for POPLIN-P solver (ablation) | CDPO · MuJoCo tasks | return | No clear effect |
| replace PSRL sampled model with LS reference model and expected-value update | CDPO vs PSRL · N-Chain MDP (N=8, 15) | regret | Improved |
| CDPO versus PSRL and HUCRL with nonlinear models | CDPO · half-cheetah | asymptotic return | Improved |
| CDPO versus PSRL and HUCRL on low-dimensional task | CDPO · inverted pendulum | asymptotic return | No clear effect |
| CDPO versus MBPO, SLBO, ME-TRPO, SAC, PPO, MPO | CDPO · MuJoCo tasks | return and sample efficiency | Improved |
Closest papers by meaning
- Minimax Optimal Reinforcement Learning with Quasi-Optimism (ICLR 2025 · similarity 0.89)
- SOMBRL: Scalable and Optimistic Model-Based RL (NeurIPS 2025 · similarity 0.89)
- Optimistic Model Rollouts for Pessimistic Offline Policy Optimization (AAAI 2024 · similarity 0.88)
- Efficient Model-Based Reinforcement Learning Through Optimistic Thompson Sampling (ICLR 2025 · similarity 0.88)
- OCEAN-MBRL: Offline Conservative Exploration for Model-Based Offline Reinforcement Learning (AAAI 2024 · similarity 0.88)
- COPlanner: Plan to Roll Out Conservatively but to Explore Optimistically for Model-Based RL (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
