Research map / Reinforcement learning / MDPs and planning

Scalable Solutions for Decision-Making Systems Using Explainable Policy Representations

AAAI 2025 · Reinforcement learning › MDPs and planning · approach: Average-reward and regularized MDPs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

A doctoral thesis proposal summarizing work on scaling MDP and simple stochastic game solvers via explainable decision-tree policies (1-2-3-Go), extensions of optimistic and sound value iteration to SGs, and planned explainable policy representations for SGs.

Problem

Solving Markov Decision Processes and Simple Stochastic Games is hard to scale because state spaces grow exponentially, and existing solvers lack interpretable policies.

Approach

Learn explainable (decision-tree) policies from small MDP instances and generalize them to larger instances by mapping corresponding states and actions; extend optimistic and sound value iteration from MDPs to SGs exploiting model structure, and plan explainable policy representations for SGs.

Key results

• Summarizes prior work: 1-2-3-Go improves scalability on several MDP benchmarks in initial experiments (no numbers given in this summary).
• PTBVI extends OVI to efficiently solve SGs with long chains of SCCs; SVI is extended to SGs with empirical gains on probabilistic loops.

Limitations

• Explainable policy representations for SGs are still under development and not evaluated here.
• 1-2-3-Go generalizability across diverse benchmarks is not yet established.
• Existing SG solvers are often incomparable, each excelling in different scenarios.

Experimental setup

Builds on: dtcontrol Optimistic Value Iteration Sound Value Iteration

Closest papers by meaning

• Explainable Reinforcement Learning via Model Transforms (NeurIPS 2022 · similarity 0.87)
• Generation of Policy-Level Explanations for Reinforcement Learning (AAAI 2019 · similarity 0.86)
• Scalable Policy-Based RL Algorithms for POMDPs (NeurIPS 2025 · similarity 0.86)
• SPOT: Scalable Policy Optimization with Trees for Markov Decision Processes (NeurIPS 2025 · similarity 0.86)
• RL in Latent MDPs is Tractable: Online Guarantees via Off-Policy Evaluation (NeurIPS 2024 · similarity 0.85)
• Approximating Shapley Explanations in Reinforcement Learning (NeurIPS 2025 · similarity 0.85)