AI Paper Atlas

Research map / Reinforcement learning / Offline RL

Instrumental Variable Value Iteration for Causal Offline Reinforcement Learning

Luofeng Liao, Zuyue Fu, Zhuoran Yang, Yixin Wang, Dingli Ma, Mladen Kolar, Zhaoran Wang

JMLR 2024 · Reinforcement learning › Offline RL · approach: Pessimism and uncertainty in offline RL

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use instrumental-variable estimation instead of ordinary regression for the transition modelIVVI · Parametric linear CMDP-IV (dim 5)regret: 590.19 -> 14.84Improved
use instrumental-variable estimation instead of ordinary regression for the transition modelIVVI · Parametric linear CMDP-IV (dim 20)regret: 2257.49 -> 505.63Improved
use instrumental-variable estimation instead of ordinary regression for the transition modelIVVI · Nonparametric CMDP-IV (dim 5)regret: 74.74 -> 35.90Improved
increase instrument strength (Sigma_z 0.5 -> 1.5) (ablation)IVVI · Parametric linear CMDP-IV (dim 5)regret: 14.84 -> 1.44Improved
increase instrument strength (Sigma_z 0.5 -> 1.5) (ablation)IVVI · Parametric linear CMDP-IV (dim 20)regret: 505.63 -> 117.20Improved
increase instrument strength (Sigma_z 0.5 -> 1.5) (ablation)IVVI · Nonparametric CMDP-IV (dim 5)regret: 35.90 -> 19.98Improved
use general covariance matrices for noise and action (robustness check) (ablation)IVVI · Nonparametric CMDP-IV (dim 3)regret: 47.85 -> 13.25Improved
use general covariance matrices for noise and action (robustness check) (ablation)IVVI · Parametric linear CMDP-IV (dim 10)regret: 832.92 -> 100.79Improved
increase state dimension from 5 to 20 (ablation)IVVI · Parametric linear CMDP-IV (config III)regret: 1.44 -> 117.20Hurt
increase state dimension from 1 to 5 (ablation)IVVI · Nonparametric CMDP-IV (config III)regret: 1.73 -> 19.98Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy