Research map / Reinforcement learning / MDPs and planning
Policy-Conditioned Uncertainty Sets for Robust Markov Decision Processes
Andrea Tirinzoni, Marek Petrik, Xiangli Chen, Brian Ziebart
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use marginally-constrained uncertainty sets instead of s,a-rectangular sets | Robust MDP policy · Invasive species, M=50, p=0.3 | negative expected return: 117.15 ± 6.76 -> 98.36 ± 0.78 | Improved |
| use marginally-constrained uncertainty sets instead of s,a-rectangular sets | Robust MDP policy · Invasive species, M=100, p=0.5 | negative expected return: 136.24 ± 8.41 -> 115.04 ± 6.18 | Improved |
| use marginally-constrained approach instead of MLE dynamics | Policy from estimated dynamics · Invasive species, M=50, p=0.3 | negative expected return: 140.36 ± 1.28 -> 98.36 ± 0.78 | Improved |
| use marginally-constrained approach instead of fitted Q-iteration | Batch policy learning · Gridworld, large grids | expected return under true dynamics | Improved |
| use marginally-constrained approach instead of s,a-rectangular uncertainty | Robust policy · Gridworld, increasing grid size | expected return under true dynamics | Improved |
| rectangular vs marginal constraints on small grids (N <= 7) | All methods · Gridworld, small grids | expected return | No clear effect |
| increase number of discrete belief states (ablation) | MC dynamic program · 5x5 gridworld | approximation error | Improved |
| increase trajectories from 50 to 100 (ablation) | MC · Invasive species, p=0.3 | negative expected return: 98.36 ± 0.78 -> 96.20 ± 0.95 | Improved |
Closest papers by meaning
- Efficient Value Iteration for s-rectangular Robust Markov Decision Processes (ICML 2024 · similarity 0.91)
- On the Complexity of Discounted Robust MDPs with $L_p$ Uncertainty Sets (NeurIPS 2026 · similarity 0.90)
- Provable Policy Gradient for Robust Average-Reward MDPs Beyond Rectangularity (ICML 2025 · similarity 0.90)
- Soft Robust MDPs and Risk-Sensitive MDPs: Equivalence, Policy Gradient, and Sample Complexity (ICLR 2024 · similarity 0.89)
- Solving Non-rectangular Reward-Robust MDPs via Frequency Regularization (AAAI 2024 · similarity 0.89)
- A Single-Loop Robust Policy Gradient Method for Robust Markov Decision Processes (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
