Research map / Reinforcement learning / Policy gradient and actor-critic
Bootstrap Estimated Uncertainty of the Environment Model for Model-Based Reinforcement Learning
Wenzhen Huang, Junge Zhang, Kaiqi Huang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add per-depth bootstrapped environment modules with prior | TreeQN-2 · Atari (9 games) | best mean score: Amidar 1170 -> 2059; Q*Bert 15970 -> 27285 | Improved |
| add bootstrapped modules | TreeQN-2 · Atari Seaquest | best mean score: 8241 -> 6415 | Hurt |
| bootstrap modules per depth rather than the entire TreeQN (ablation) | Bootstrapped TreeQN · Atari (9 games) | best mean score: Seaquest 4610 -> 6415; Amidar 1188 -> 2059 | Improved |
| sample target values from bootstrap distribution instead of fixed target modules (ablation) | Bootstrapped TreeQN · Atari (9 games) | best mean score: Frostbite 4279 vs 2889 is reversed; Krull 9590 -> 12010; Amidar 1527 -> 2059 | Improved |
| add frozen random prior network (ablation) | Bootstrapped TreeQN · Atari (9 games) | best mean score: Amidar 1888 -> 2059; Frostbite 6039 vs 2889; Seaquest 7313 vs 6415 | Mixed |
| use bootstrapped modules | TreeQN · Box Pushing | reward prediction loss | Improved |
| use bootstrapped modules | TreeQN · Box Pushing | average reward | No clear effect |
| double embedding size of TreeQN (wide) instead of adding bootstrap | TreeQN-2 · Atari (9 games) | best mean score: Alien 2497 -> 2709; Seaquest 8241 -> 8956 | Mixed |
Closest papers by meaning
- The Edge-of-Reach Problem in Offline Model-Based Reinforcement Learning (NeurIPS 2024 · similarity 0.88)
- Any-step Dynamics Model Improves Future Predictions for Online and Offline Reinforcement Learning (ICLR 2025 · similarity 0.88)
- OCEAN-MBRL: Offline Conservative Exploration for Model-Based Offline Reinforcement Learning (AAAI 2024 · similarity 0.87)
- COPlanner: Plan to Roll Out Conservatively but to Explore Optimistically for Model-Based RL (ICLR 2024 · similarity 0.87)
- Adversarial Diffusion for Robust Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
- SUMO: Search-Based Uncertainty Estimation for Model-Based Offline Reinforcement Learning (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
