Research map / Reinforcement learning / Offline RL
Model-Based Offline Reinforcement Learning with Local Misspecification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Select policies by maximizing the local-misspecification lower bound instead of using MOPO directly | D4RL Hopper and HalfCheetah policy selection · D4RL tasks in the results table | return | Helped |
| Use MBLB with a quadratic test-function family rather than the strongest baseline | D4RL medium Hopper policy selection · D4RL medium Hopper | return: 379.4 -> 808.5 | Helped |
| Use MBLB with a linear test-function family rather than the strongest baseline | D4RL med-expert Hopper policy selection · D4RL med-expert Hopper | return: 183.8 -> 261.1 | Helped |
| Use MBLB with a quadratic test-function family rather than the strongest baseline | D4RL expert Hopper policy selection · D4RL expert Hopper | return: 93.8 -> 121.0 | Helped |
| Use MBLB with a linear test-function family rather than the strongest baseline | D4RL medium HalfCheetah policy selection · D4RL medium HalfCheetah | return: 3858.2 -> 3290.4 | Hurt |
| Use MBLB with a linear test-function family rather than the strongest baseline | D4RL med-expert HalfCheetah policy selection · D4RL med-expert HalfCheetah | return: -77.0 -> 207.4 | Helped |
| Use a quadratic rather than linear test-function family (ablation) | MBLB · D4RL Hopper and HalfCheetah | return: 591.7 -> 808.5; 261.1 -> 242.5; 118.2 -> 121.0; 3290.4 -> 2484.2; 207.4 -> 192.8 | Mixed |
| Jointly select the policy and dynamics model instead of learning dynamics first and planning afterward | Model-based offline policy selection · LQR with misspecified transition classes | policy selection quality | Helped |
| Use GAE-based value estimation instead of standard TD-1 FQE (ablation) | MBLB policy evaluation · D4RL policy evaluation | policy evaluation estimation quality | Helped |
| Lower the cutoff threshold (ablation) | MBLB · LQR | lower-bound quality | Hurt |
| Reduce the number of trajectories (ablation) | MBLB · LQR | policy selection quality | Hurt |
Closest papers by meaning
- A Unified Principle of Pessimism for Offline Reinforcement Learning under Model Mismatch (NeurIPS 2024 · similarity 0.92)
- Model Selection for Off-policy Evaluation: New Algorithms and Experimental Protocol (NeurIPS 2025 · similarity 0.92)
- The Edge-of-Reach Problem in Offline Model-Based Reinforcement Learning (NeurIPS 2024 · similarity 0.91)
- A Clean Slate for Offline Reinforcement Learning (NeurIPS 2025 · similarity 0.91)
- Reward-Consistent Dynamics Models are Strongly Generalizable for Offline Reinforcement Learning (ICLR 2024 · similarity 0.91)
- Minimax Optimal and Computationally Efficient Algorithms for Distributionally Robust Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.