Research map / Reinforcement learning / Policy gradient and actor-critic

Simultaneous Statistical Inference for Off-Policy Evaluation in Reinforcement Learning

NeurIPS 2025 · Reinforcement learning › Policy gradient and actor-critic · approach: Importance-weighted OPE estimators

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a simultaneous statistical inference framework for off-policy evaluation that gives valid uniform confidence bands over a continuous initial state space, using sieve-based Q-function estimation.

Problem

Existing off-policy evaluation methods give point estimates or pointwise confidence intervals, which do not quantify global uncertainty across an infinite or continuous initial state space.

Approach

Uses sieve-based Q-function estimation to build simultaneous (uniform) confidence bands for the value function over the entire initial state space, providing theoretically justified global inference for OPE.

Closest papers by meaning

• Bootstrapping Fitted Q-Evaluation for Off-Policy Inference (ICML 2021 · similarity 0.91)
• More Efficient Off-Policy Evaluation through Regularized Targeted Learning (ICML 2019 · similarity 0.91)
• Non-asymptotic Confidence Intervals of Off-policy Evaluation: Primal and Dual Bounds (ICLR 2021 · similarity 0.91)
• Dynamic Uncertainty Estimation for Offline Reinforcement Learning (AAAI 2025 · similarity 0.89)
• Statistical Tractability of Off-policy Evaluation of History-dependent Policies in POMDPs (ICLR 2025 · similarity 0.89)
• Conformal Prediction Beyond the Horizon: Distribution-Free Inference for Policy Evaluation (NeurIPS 2025 · similarity 0.88)