Research map / Reinforcement learning / MDPs and planning
A Theory of Regularized Markov Decision Processes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply a strongly convex regularizer to the Bellman operators | Regularized MDPs · Regularized MDPs | Bellman operator contraction factor: Both operators are γ-contractions in supremum norm. | Helped |
| Solve the regularized MDP and use its optimal policy in the original MDP | Regularized MDPs · Original MDP | optimality gap: v_* - \frac{U_\Omega - L_\Omega}{1-\gamma} \leq v_{\pi_{*,\Omega}} \leq v_* | Hurt |
| Use Bregman divergence to regularize greedy updates between consecutive policies | MD-MPI types 1 and 2 · MD-MPI, exact case | average regret: \frac 1 K \|L_K\|_\infty \leq \frac{1-\gamma^K}{(1-\gamma)^2} \frac{2 \gamma \|v_* - v_0\|_\infty + R_{\Omega_{\pi_0}}}{ | Helped |
| Use a fixed regularizer in modified policy iteration | Regularized MPI · Regularized MPI | error propagation bound: same bound (up to the fact that it deals with regularized MDPs) | No clear effect |
| Replace a fixed regularizer with a Bregman divergence to the previous policy | MD-MPI · MD-MPI, exact case | convergence rate: linear convergence rate; logarithmic convergence rate; square dependency in \frac{1}{1-\gamma} instead of linear | Hurt |
| Select the best policy computed so far rather than the last policy | MD-MPI · MD-MPI, approximate case | loss: \min_{1\leq k \leq K} \|v_* - v_{\pi_k}\|_{1,\rho} \leq \frac 1 K \|L_K\|_{p,\rho} | Helped |
Closest papers by meaning
- A Regularized Approach to Sparse Optimal Policy in Reinforcement Learning (NeurIPS 2019 · similarity 0.93)
- Regularization Guarantees Generalization in Bayesian Reinforcement Learning through Algorithmic Stability (AAAI 2022 · similarity 0.91)
- Doubly Regularized Markov Decision Processes for Robust Reinforcement Learning (ICML 2026 · similarity 0.91)
- Natural Policy Gradient as Doubly Smoothed Policy Iteration: A Bellman-Operator Framework (NeurIPS 2026 · similarity 0.90)
- Convergence Theorems for Entropy-Regularized and Distributional Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
- Global linear convergence of entropy-regularized softmax policy gradient beyond tabular MDPs (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.