Research map / Reinforcement learning / Policy gradient and actor-critic
Control-Oriented Model-Based Reinforcement Learning with Implicit Differentiation
Evgenii Nikishin, Romina Abachi, Rishabh Agarwal, Pierre-Luc Bacon
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Optimize the model for downstream returns rather than likelihood | OMD versus MLE · 2-state, 2-action tabular MDP under model misspecification | expected return | Improved |
| Optimize the model for downstream returns rather than next-state prediction | OMD versus MLE · CartPole with limited model capacity | return | Improved |
| Optimize the model for downstream returns rather than next-state prediction | OMD versus MLE · CartPole with uninformative state distractors | return | Improved |
| Optimize the model for downstream returns rather than MSE | OMD versus MLE · HalfCheetah-v2 under model misspecification | return | Improved |
| Use K=3 or K=10 instead of K=1 inner-loop steps (ablation) | OMD · CartPole | return | No clear effect |
| Add double Q-learning (ablation) | OMD · CartPole | return | Improved |
| Use the inverse Jacobian instead of the identity approximation (ablation) | OMD · CartPole | return | No clear effect |
Closest papers by meaning
- Model-based Policy Optimization with Unsupervised Model Adaptation (NeurIPS 2020 · similarity 0.88)
- Maximum Likelihood Reinforcement Learning (ICML 2026 · similarity 0.88)
- Sample-Efficient Reinforcement Learning via Conservative Model-Based Actor-Critic (AAAI 2022 · similarity 0.88)
- Inverting the Bellman Equation: From $Q$-Values to World Models (NeurIPS 2026 · similarity 0.88)
- Model-Based Offline Reinforcement Learning with Local Misspecification (AAAI 2023 · similarity 0.88)
- Deterministic Uncertainty Propagation for Improved Model-Based Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
