Research map / Reinforcement learning / Policy gradient and actor-critic
Sample-Efficient Reinforcement Learning via Conservative Model-Based Actor-Critic
Zhihai Wang, Jie Wang, Qi Zhou, Bin Li, Houqiang Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use CMBAC instead of MBPO | model-based control · Humanoid | sample efficiency: 200 thousand steps matches that of MBPO at 300 thousand steps | Improved |
| use CMBAC instead of MBPO | model-based control · Walker2d-NT and Hopper-NT | average return | Improved |
| use CMBAC instead of MBPO and MOPO-Online | model-based control with Gaussian action noise · Walker2d and HalfCheetah noisy environments | sample efficiency: 100k steps for HalfCheetah and 200k steps for Walker2d | Improved |
| increase Q-network size (ablation) | MBPO · Ant and Walker2d | average return | No clear effect |
| learn multiple Q-value estimates from ensemble models (ablation) | B-MBPO · Ant and Walker2d | average return | Improved |
| add conservative policy optimization using the bottom-k Q-value estimates (ablation) | B-LMEQ-MBPO · Ant | average return | Improved |
| add conservative policy optimization using the bottom-k Q-value estimates (ablation) | B-LMEQ-MBPO · Walker2d | average return | Hurt |
| increase Q-network size (ablation) | MBPO · Hopper | average return | No clear effect |
| learn multiple Q-value estimates from ensemble models (ablation) | B-MBPO · Hopper | average return | Improved |
| add conservative policy optimization using the bottom-k Q-value estimates (ablation) | B-LMEQ-MBPO · Hopper | average return | Improved |
| add conservative policy optimization using the bottom-k Q-value estimates (ablation) | B-LMEQ-MBPO · HalfCheetah | asymptotic performance | Hurt |
| set the number of probabilistic networks per model to M=2 (ablation) | CMBAC · Ant and Walker2d | average return | Improved |
Closest papers by meaning
- Actor-Critics Can Achieve Optimal Sample Efficiency (ICML 2025 · similarity 0.91)
- Improving Sample Complexity Bounds for (Natural) Actor-Critic Algorithms (NeurIPS 2020 · similarity 0.90)
- Model-Augmented Actor-Critic: Backpropagating through Paths (ICLR 2020 · similarity 0.90)
- Langevin Soft Actor-Critic: Efficient Exploration through Uncertainty-Driven Critic Learning (ICLR 2025 · similarity 0.88)
- Finite-Time Convergence and Sample Complexity of Actor-Critic Multi-Objective Reinforcement Learning (ICML 2024 · similarity 0.88)
- Actor-Free Continuous Control via Structurally Maximizable Q-Functions (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
