Research map / Reinforcement learning / Policy gradient and actor-critic
Mismatched No More: Joint Model-Policy Optimization for Model-Based RL
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace maximum-likelihood model with GAN-like MnM model objective | MBPO · ROBEL manipulation (4 tasks) | return: p \le 0.03 | Helped |
| replace maximum-likelihood model with GAN-like MnM model objective | MBPO · OpenAI Gym locomotion (3 tasks) | return | No clear effect |
| train model and policy with the MnM lower bound instead of the VMBPO upper-bound objective | MnM-approx · OpenAI Gym locomotion (3 tasks) | return: p = 0.04, 0.00, 0.03 | Helped |
| remove the classifier term from the augmented reward (ablation) | MnM · gridworld with low-resolution model | goal reaching | Hurt |
| add the classifier term in the online continuous-control setting (ablation) | MnM · online continuous control | return | Hurt |
| use MnM learned optimistic dynamics instead of true model with Q-learning | Q-learning · 10x10 stochastic gridworld | convergence speed | Helped |
| use MnM log-transformed reward instead of VMBPO risk-seeking exponentiated objective | VMBPO · 3-state stochastic MDP | expected reward | Helped |
| replace VAML or value-weighted MLE model learning with MnM-approx | MBPO · DClawScrewFixed-v0 | return | Helped |
| increase model rollout horizon (ablation) | MnM · continuous control task | return | Hurt |
| compare Q-values of MnM-approx and MBPO | MBPO · metaworld-drawer-open-v2 | Q-value stability | Helped |
Closest papers by meaning
- The Virtues of Laziness in Model-based RL: A Unified Objective and Algorithms (ICML 2023 · similarity 0.91)
- Towards General-Purpose Model-Free Reinforcement Learning (ICLR 2025 · similarity 0.91)
- Constrained Reinforcement Learning Under Model Mismatch (ICML 2024 · similarity 0.90)
- The Edge-of-Reach Problem in Offline Model-Based Reinforcement Learning (NeurIPS 2024 · similarity 0.89)
- Dual RL: Unification and New Methods for Reinforcement and Imitation Learning (ICLR 2024 · similarity 0.89)
- PWM: Policy Learning with Multi-Task World Models (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.