AI Paper Atlas

Research map / Reinforcement learning / MDPs and planning

Temporal Regularization for Markov Decision Process

Pierre Thodoroff, Audrey Durand, Joelle Pineau, Doina Precup

NeurIPS 2018 · Reinforcement learning › MDPs and planning · approach: POMDPs and confounded RL

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add temporal regularization to value estimatesPolicy evaluation on a synthetic 3-state MDP · Synthetic 3-state MDPvariance of value estimatesImproved
Add temporal regularizationValue learning in the two-room MDP · Two-room MDPvalue-estimation errorImproved
Use exponential smoothing instead of only the previous state estimate (ablation)Temporally regularized value learning · Two-room MDPvalue-estimation errorImproved
Add temporal regularizationLinear TD value estimation · Noisy continuous walkparameter-estimation errorImproved
Increase exponential-smoothing parameter λ (ablation)Temporally regularized value estimation · Noisy continuous walkabsolute value-estimation errorMixed
Add exponential temporal regularization to the critic targetPPO · Atari gamesnormalized performanceImproved
Increase the number of temporally nearby states whose rewards are smoothed (ablation)Temporally regularized policy evaluation · Synthetic 10-state MDPbias from the unregularized value functionImproved
Use an intermediate regularization parameter β rather than an extreme value (ablation)Temporally regularized Markov chain · Synthetic 10-state MDPmixing timeImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy