Research map / Reinforcement learning / Offline RL
Autoregressive Dynamics Models for Offline Policy Evaluation and Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Model next-state dimensions sequentially conditioned on previous dimensions instead of assuming conditional independence | Dynamics models for DM Control tasks · RL Unplugged, 9 DM Control tasks, heldout validation | negative log-likelihood | Helped |
| Use autoregressive rather than feedforward dynamics for policy evaluation | Model-based OPE · RL Unplugged, 9 tasks | median Spearman rank correlation: 0.64 -> 0.90 | Helped |
| Use autoregressive rather than feedforward dynamics for policy evaluation | Model-based OPE · RL Unplugged, 9 tasks | median normalized Regret@5: 0.24 -> 0.04 | Helped |
| Use autoregressive rather than feedforward dynamics for policy evaluation | Model-based OPE · RL Unplugged, 9 tasks | median average absolute error: 30.12 -> 9.71 | Helped |
| Use autoregressive dynamics instead of feedforward dynamics for planning | CRR policy optimization · Offline policy optimization environments tested | policy performance | Helped |
| Add model-based planning (ablation) | CRR AR with autoregressive-model augmented data · Offline policy optimization environments tested | policy performance | Helped |
| Add autoregressive-model data augmentation (ablation) | CRR exp · Cheetah run | policy performance | Helped |
| Add autoregressive-model data augmentation (ablation) | CRR exp · Cartpole swingup, Fish swim, Finger turn hard | policy performance | No clear effect |
| Combine data augmentation with planning (ablation) | CRR with autoregressive dynamics · Cheetah run, Fish swim | policy return | Helped |
Closest papers by meaning
- Reward-Consistent Dynamics Models are Strongly Generalizable for Offline Reinforcement Learning (ICLR 2024 · similarity 0.89)
- Forecasting in Offline Reinforcement Learning for Non-stationary Environments (NeurIPS 2025 · similarity 0.89)
- Model Selection for Off-policy Evaluation: New Algorithms and Experimental Protocol (NeurIPS 2025 · similarity 0.88)
- Optimistic Model Rollouts for Pessimistic Offline Policy Optimization (AAAI 2024 · similarity 0.88)
- Any-step Dynamics Model Improves Future Predictions for Online and Offline Reinforcement Learning (ICLR 2025 · similarity 0.88)
- Offline Transition Modeling via Contrastive Energy Learning (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.