AI Paper Atlas

Research map / Reinforcement learning / MDPs and planning

Representation Balancing MDPs for Off-policy Policy Evaluation

Yao Liu, Omer Gottesman, Aniruddh Raghu, Matthieu Komorowski, Aldo A Faisal, Finale Doshi-Velez, Emma Brunskill

NeurIPS 2018 · Reinforcement learning › MDPs and planning · approach: Offline RL sample complexity · 8 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use representation-balanced MDP fitting instead of empirical-risk model fittingOPPE model estimator · Cart Pole long horizonmean root MSE: 0.7535 -> 0.4121Improved
use representation-balanced MDP fitting instead of empirical-risk model fittingOPPE model estimator · Cart Pole long horizonindividual root MSE: 1.313 -> 1.033Improved
use representation-balanced MDP fitting instead of empirical-risk model fittingOPPE model estimator · Cart Pole short horizonmean root MSE: 0.1254 -> 0.07836Hurt
use representation-balanced MDP fitting instead of empirical-risk model fittingOPPE model estimator · Cart Pole short horizonindividual root MSE: 0.5506 -> 0.4811Improved
use representation-balanced MDP fitting instead of empirical-risk model fittingOPPE model estimator · Mountain Carmean root MSE: 17.15 -> 12.31Improved
use representation-balanced MDP fitting instead of empirical-risk model fittingOPPE model estimator · Mountain Carindividual root MSE: 36.36 -> 31.38Improved
use representation-balanced MDP fitting instead of empirical-risk model fittingOPPE model estimator · HIV simulatormean relative root MSE: 0.067 -> 0.062Improved
add the MSEμ term to the model-fitting objective (ablation)RepBM · Cart Pole short horizonindividual root MSE: 0.5974 -> 0.4811Improved
add the MSEμ term to the model-fitting objective (ablation)RepBM · Mountain Carmean root MSE: 72.61 -> 12.31Improved
add the MSEμ term to the model-fitting objective (ablation)RepBM · Mountain Carindividual root MSE: 79.46 -> 31.38Improved
vary the IPM penalty weight α (ablation)RepBM · Cart Pole and Mountain Carroot MSENo clear effect
add non-zero IPM representation balancing (ablation)RepBM · Cart Pole and Mountain Carroot MSEImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy