AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

The Surprising Difficulty of Search in Model-Based Reinforcement Learning

Wei-Di Chang, Mikael Henaff, Brandon Amos, Gregory Dudek, Scott Fujimoto

ICML 2026 · Reinforcement learning › Policy gradient and actor-critic · approach: Model-based RL with guarantees · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add MPC to MR.QMR.Q · Gymnormalized score: 1.46 -> 0.67Hurt
add MPC to MR.QMR.Q · DMCaverage return / 1k: 0.84 -> 0.65Hurt
add MPC to MR.QMR.Q · HumanoidBench with handaverage return / 1k: 0.31 -> 0.38Improved
use MRS.Q (min over 10 value functions plus MPC) instead of MR.Q + MPCMR.Q + MPC · HumanoidBench with handaverage return / 1k: 0.38 -> 0.58Improved
use 10 value functions instead of 2 (ablation)MRS.Q · Gymnormalized score difference: -0.63Improved
use 10 value functions instead of 5 (ablation)MRS.Q · Gymnormalized score difference: -0.37Improved
use 20 value functions instead of 10 (ablation)MRS.Q · HumanoidBench no handperformance difference: 0.03No clear effect
take min over the full ensemble instead of two sampled members (ablation)MRS.Q · Gymnormalized score difference: -0.49Improved
take min over full ensemble during MPC instead of mean (ablation)MRS.Q · Gymnormalized score difference: -0.33Improved
remove exploration noise (ablation)MRS.Q · Gymnormalized score difference: -0.13Improved
use SEM instead of ELU (ablation)MRS.Q · Gymnormalized score difference: -0.28Improved
use SEM instead of ELU (ablation)MRS.Q · HumanoidBench with handperformance difference: 0.10Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy