Research map / Reinforcement learning / Visual and representation RL
Reinforcement Learning in Presence of Discrete Markovian Context Evolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the contextual model with SAC rather than a context-agnostic SAC baseline | SAC · Drone | expected return: -0.80 ± 3.08 -> 28.41 ± 1.16 | Helped |
| Use the contextual model with SAC rather than a context-agnostic SAC baseline | SAC · Intersection | expected return: 499.62 ± 19.98 -> 555.11 ± 20.21 | Helped |
| Use contextual SAC instead of full-information SAC | Cart-Pole Swing-Up, hard failure · hard failure | expected return: 84.50 ± 1.79 -> 85.38 ± 1.64 | No clear effect |
| Use the HDP prior rather than maximum-likelihood model learning | Cart-Pole Swing-Up context model · Cart-Pole Swing-Up | context model accuracy and interpretability | Helped |
| Add context distillation during training (ablation) | HDP-C-MDP · Cart-Pole Swing-Up | stationary probability of third most probable context: 2.25e-02 -> 9.86e-03 | Helped |
| Increase the distillation threshold from 0 to 0.01 (ablation) | HDP-C-MDP with K=20 · Cart-Pole Swing-Up | stationary probability of third most probable context: 2.25e-02 -> 1.60e-02 | Helped |
| Increase the distillation threshold from 0.01 to 0.1 (ablation) | HDP-C-MDP with K=20 · Cart-Pole Swing-Up | stationary probability of third most probable context: 1.60e-02 -> 9.86e-03 | Helped |
| Overestimate the context cardinality while using distillation (ablation) | HDP-C-MDP · Cart-Pole Swing-Up | stationary probability of third most probable context: 9.86e-03 | No clear effect |
| Use C-CEM instead of C-SAC | Contextual control · Cart-Pole Swing-Up, hard failure | expected return: 85.38 ± 1.64 -> 87.63 ± 0.14 | Helped |
| Use C-CEM instead of C-SAC | Contextual control · Intersection | expected return: 555.11 ± 20.21 -> 529.75 ± 78.12 | Hurt |
Closest papers by meaning
- Zero-Shot Context Generalization in Reinforcement Learning from Few Training Contexts (NeurIPS 2025 · similarity 0.89)
- Balancing Context Length and Mixing Times for Reinforcement Learning at Scale (NeurIPS 2024 · similarity 0.89)
- Zero-shot Model-based Reinforcement Learning using Large Language Models (ICLR 2025 · similarity 0.89)
- Towards Large-Scale In-Context Reinforcement Learning by Meta-Training in Randomized Worlds (NeurIPS 2025 · similarity 0.89)
- XLand-100B: A Large-Scale Multi-Task Dataset for In-Context Reinforcement Learning (ICLR 2025 · similarity 0.88)
- Distilling Reinforcement Learning Algorithms for In-Context Model-Based Planning (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.