Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Robust and Efficient Transfer Learning with Hidden Parameter Markov Decision Processes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Embed instance-specific latent parameters as inputs to the transition model, allowing nonlinear interactions with state | HiP-MDP · 2D navigation | policy learning performance | Helped |
| Embed instance-specific latent parameters as inputs to the transition model, allowing nonlinear interactions with state | HiP-MDP · Acrobot | episodes to learn an optimal policy: after a single episode | Helped |
| Embed instance-specific latent parameters as inputs to the transition model, allowing nonlinear interactions with state | HiP-MDP · HIV treatment | episodes to learn an optimal policy: after a single episode | Helped |
| Use embedded rather than linearly combined latent parameters (ablation) | HiP-MDP · HIV treatment | policy learning performance | Helped |
| Use embedded rather than linearly combined latent parameters (ablation) | HiP-MDP · 2D navigation | transition prediction accuracy | Helped |
| Use embedded rather than linearly combined latent parameters (ablation) | HiP-MDP · Acrobot | policy consistency | Helped |
| Use a linear combination of latent parameters instead of embedding them in the transition-model input (ablation) | HiP-MDP · HIV treatment | policy learning performance | Hurt |
| Train the transition model from scratch on the new instance rather than transferring a pretrained model (ablation) | Model-based from scratch · 2D navigation | policy learning performance | Hurt |
| Use a single average transition model across instances instead of instance-specific latent representations (ablation) | Average model · 2D navigation | policy learning performance | Hurt |
| Learn a policy directly from environment observations instead of using a transferred model (ablation) | Model-free DDQN · 2D navigation | policy learning performance: first 10 episodes | Hurt |
| Replace GP basis functions with a BNN transition model (ablation) | HiP-MDP · 2D navigation | training time: a little more than 8 hours vs. close to 70 hours | Helped |
| Update latent parameters and BNN parameters in small alternating steps rather than updating either too aggressively (ablation) | HiP-MDP · Transfer performance | transfer performance | Helped |
Closest papers by meaning
- Performance Bounds for Model and Policy Transfer in Hidden-parameter MDPs (ICLR 2023 · similarity 0.86)
- Learning Robust State Abstractions for Hidden-Parameter Block MDPs (ICLR 2021 · similarity 0.86)
- Generalized Hidden Parameter MDPs:Transferable Model-Based RL in a Handful of Trials (AAAI 2020 · similarity 0.84)
- Hidden Parameter Recurrent State Space Models For Changing Dynamics Scenarios (ICLR 2022 · similarity 0.84)
- Online Bayesian Transfer Learning for Sequential Data Modeling (ICLR 2017 · similarity 0.84)
- Transfer Q-Learning with Composite MDP Structures (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.