Research map / Reinforcement learning / MDPs and planning
Temporal Regularization for Markov Decision Process
Pierre Thodoroff, Audrey Durand, Joelle Pineau, Doina Precup
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add temporal regularization to value estimates | Policy evaluation on a synthetic 3-state MDP · Synthetic 3-state MDP | variance of value estimates | Improved |
| Add temporal regularization | Value learning in the two-room MDP · Two-room MDP | value-estimation error | Improved |
| Use exponential smoothing instead of only the previous state estimate (ablation) | Temporally regularized value learning · Two-room MDP | value-estimation error | Improved |
| Add temporal regularization | Linear TD value estimation · Noisy continuous walk | parameter-estimation error | Improved |
| Increase exponential-smoothing parameter λ (ablation) | Temporally regularized value estimation · Noisy continuous walk | absolute value-estimation error | Mixed |
| Add exponential temporal regularization to the critic target | PPO · Atari games | normalized performance | Improved |
| Increase the number of temporally nearby states whose rewards are smoothed (ablation) | Temporally regularized policy evaluation · Synthetic 10-state MDP | bias from the unregularized value function | Improved |
| Use an intermediate regularization parameter β rather than an extreme value (ablation) | Temporally regularized Markov chain · Synthetic 10-state MDP | mixing time | Improved |
Closest papers by meaning
- A Theory of Regularized Markov Decision Processes (ICML 2019 · similarity 0.89)
- Doubly Regularized Markov Decision Processes for Robust Reinforcement Learning (ICML 2026 · similarity 0.88)
- Flowing Through States: Neural ODE Regularization for Reinforcement Learning (ICLR 2026 · similarity 0.88)
- Regularization Guarantees Generalization in Bayesian Reinforcement Learning through Algorithmic Stability (AAAI 2022 · similarity 0.88)
- Learning to Trust Bellman Updates: Selective State-Adaptive Regularization for Offline RL (ICML 2025 · similarity 0.86)
- The surprising efficiency of temporal difference learning for rare event prediction (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
