Research map / Reinforcement learning / MDPs and planning
Model-based Reinforcement Learning for Semi-Markov Decision Processes with Neural ODEs
Jianzhun Du, Joseph Futoma, Finale Doshi-Velez
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Latent-ODE dynamics model instead of vanilla RNN | model-based RL on HIV · HIV | cumulative reward (x10^7): 0.78 -> 30.32 | Improved |
| use ODE-RNN instead of vanilla RNN | model-based RL on windy gridworld · Windy gridworld | cumulative reward: -54.02 -> -58.50 | Hurt |
| use ODE-RNN instead of vanilla RNN | model-based RL on Acrobot · Acrobot | cumulative reward: -179.35 -> -65.90 | Improved |
| lower state prediction error does not imply better policy (Delta t-RNN vs Latent-ODE) | Delta t-RNN · Windy gridworld | state prediction error vs reward: 0.334 vs 0.845 error; -45.64 vs -34.87 reward | Mixed |
| transfer pretrained Latent-ODE to new regular time schedule without retraining | Latent-ODE · Acrobot tau=0.2 | cumulative reward: -54.26 -> -171.37 | Hurt |
| add latent variable z to policy (ablation) | Latent-ODE model-based policy · Partially observable HIV | reward (x10^7): 12.88 -> 18.04 | Improved |
| optimize interval times with model-based schedule | HIV Latent-ODE · HIV | interaction rate at similar return | Improved |
Closest papers by meaning
- Continuous-time Model-based Reinforcement Learning (ICML 2021 · similarity 0.91)
- When to Sense and Control? A Time-adaptive Approach for Continuous-Time RL (NeurIPS 2024 · similarity 0.89)
- Overcoming Slow Decision Frequencies in Continuous Control: Model-Based Sequence Reinforcement Learning for Model-Free Control (ICLR 2025 · similarity 0.88)
- Rethinking Transformers in Solving POMDPs (ICML 2024 · similarity 0.87)
- Reinforcement Learning with Random Time Horizons (ICML 2025 · similarity 0.87)
- Periodic agent-state based Q-learning for POMDPs (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
