Research map / Reinforcement learning / MDPs and planning
Does the Markov Decision Process Fit the Data: Testing for the Markov Property in Sequential Decision Making
Chengchun Shi, Runzhe Wan, Rui Song, Wenbin Lu, Ling Leng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use four decision points of history rather than one (ablation) | FQI policy on OhioT1DM · OhioT1DM | estimated value: -90.82 -> -52.57 | Improved |
| Use four decision points of history rather than two (ablation) | FQI policy on OhioT1DM · OhioT1DM | estimated value: -57.53 -> -52.57 | Improved |
| Use four decision points of history rather than three (ablation) | FQI policy on OhioT1DM · OhioT1DM | estimated value: -63.77 -> -52.57 | Improved |
| Use four decision points of history rather than five (ablation) | FQI policy on OhioT1DM · OhioT1DM | estimated value: -56.23 -> -52.57 | Improved |
| Use four decision points of history rather than six (ablation) | FQI policy on OhioT1DM · OhioT1DM | estimated value: -60.05 -> -52.57 | Improved |
| Use four decision points of history rather than seven (ablation) | FQI policy on OhioT1DM · OhioT1DM | estimated value: -63.70 -> -52.57 | Improved |
| Use four decision points of history rather than eight (ablation) | FQI policy on OhioT1DM · OhioT1DM | estimated value: -54.85 -> -52.57 | Improved |
| Use four decision points of history rather than nine (ablation) | FQI policy on OhioT1DM · OhioT1DM | estimated value: -65.08 -> -52.57 | Improved |
| Use four decision points of history rather than ten (ablation) | FQI policy on OhioT1DM · OhioT1DM | estimated value: -59.59 -> -52.57 | Improved |
| Select the history length by sequential Markov-assumption testing | FQI policy on OhioT1DM · OhioT1DM | estimated value | Improved |
Closest papers by meaning
- Sample-Efficient Learning of POMDPs with Multiple Observations In Hindsight (ICLR 2024 · similarity 0.88)
- Efficient and Sharp Off-Policy Evaluation in Robust Markov Decision Processes (NeurIPS 2024 · similarity 0.88)
- Model-Based Exploration in Monitored Markov Decision Processes (ICML 2025 · similarity 0.87)
- A PAC Learning Algorithm for LTL and Omega-Regular Objectives in MDPs (AAAI 2024 · similarity 0.87)
- Model-based Reinforcement Learning for Confounded POMDPs (ICML 2024 · similarity 0.87)
- Solving Robust Markov Decision Processes: Generic, Reliable, Efficient (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
