Research map / Reinforcement learning / MDPs and planning
Representation Balancing MDPs for Off-policy Policy Evaluation
Yao Liu, Omer Gottesman, Aniruddh Raghu, Matthieu Komorowski, Aldo A Faisal, Finale Doshi-Velez, Emma Brunskill
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use representation-balanced MDP fitting instead of empirical-risk model fitting | OPPE model estimator · Cart Pole long horizon | mean root MSE: 0.7535 -> 0.4121 | Improved |
| use representation-balanced MDP fitting instead of empirical-risk model fitting | OPPE model estimator · Cart Pole long horizon | individual root MSE: 1.313 -> 1.033 | Improved |
| use representation-balanced MDP fitting instead of empirical-risk model fitting | OPPE model estimator · Cart Pole short horizon | mean root MSE: 0.1254 -> 0.07836 | Hurt |
| use representation-balanced MDP fitting instead of empirical-risk model fitting | OPPE model estimator · Cart Pole short horizon | individual root MSE: 0.5506 -> 0.4811 | Improved |
| use representation-balanced MDP fitting instead of empirical-risk model fitting | OPPE model estimator · Mountain Car | mean root MSE: 17.15 -> 12.31 | Improved |
| use representation-balanced MDP fitting instead of empirical-risk model fitting | OPPE model estimator · Mountain Car | individual root MSE: 36.36 -> 31.38 | Improved |
| use representation-balanced MDP fitting instead of empirical-risk model fitting | OPPE model estimator · HIV simulator | mean relative root MSE: 0.067 -> 0.062 | Improved |
| add the MSEμ term to the model-fitting objective (ablation) | RepBM · Cart Pole short horizon | individual root MSE: 0.5974 -> 0.4811 | Improved |
| add the MSEμ term to the model-fitting objective (ablation) | RepBM · Mountain Car | mean root MSE: 72.61 -> 12.31 | Improved |
| add the MSEμ term to the model-fitting objective (ablation) | RepBM · Mountain Car | individual root MSE: 79.46 -> 31.38 | Improved |
| vary the IPM penalty weight α (ablation) | RepBM · Cart Pole and Mountain Car | root MSE | No clear effect |
| add non-zero IPM representation balancing (ablation) | RepBM · Cart Pole and Mountain Car | root MSE | Improved |
Closest papers by meaning
- Off-Policy Evaluation for Episodic Partially Observable Markov Decision Processes under Non-Parametric Models (NeurIPS 2022 · similarity 0.87)
- Statistical Tractability of Off-policy Evaluation of History-dependent Policies in POMDPs (ICLR 2025 · similarity 0.86)
- Efficient and Sharp Off-Policy Evaluation in Robust Markov Decision Processes (NeurIPS 2024 · similarity 0.86)
- RL in Latent MDPs is Tractable: Online Guarantees via Off-Policy Evaluation (NeurIPS 2024 · similarity 0.86)
- Q-MMR: Off-Policy Evaluation via Recursive Reweighting and Moment Matching (NeurIPS 2026 · similarity 0.86)
- Offline Data Enhanced On-Policy Policy Gradient with Provable Guarantees (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
