Research map / Reinforcement learning / Policy gradient and actor-critic

Variational Approach to Optimal IPS Estimator for Multi-logger Off-Policy Evaluation

NeurIPS 2026 · Reinforcement learning › Policy gradient and actor-critic · approach: Importance-weighted OPE estimators

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Derives a variational formulation of the optimal IPS estimator for off-policy evaluation with data from multiple logging policies, resolving the lack of a uniformly best multi-logger IPS estimator.

Problem

Off-policy evaluation in contextual bandits with data from multiple logging policies; no IPS estimator was known to consistently dominate the others in this setting (Agarwal et al., 2017).

Approach

Derives an optimal IPS estimator for the multi-logger setting via a variational approach, resolving the dilemma that no single existing IPS weighting consistently outperforms the others.

Surprising findings

• No IPS estimator was previously known to consistently outperform the others in the multi-logger setting, per Agarwal et al. (2017).

Experimental setup

Builds on: inverse propensity scoring Agarwal et al. 2017 multi-logger OPE

Closest papers by meaning

• Log-Sum-Exponential Estimator for Off-Policy Evaluation and Learning (ICML 2025 · similarity 0.92)
• Optimal and Adaptive Off-policy Evaluation in Contextual Bandits (ICML 2017 · similarity 0.91)
• Marginal Density Ratio for Off-Policy Evaluation in Contextual Bandits (NeurIPS 2023 · similarity 0.91)
• Off-Policy Evaluation for Ranking Policies under Deterministic Logging Policies (ICLR 2026 · similarity 0.90)
• Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learning (NeurIPS 2024 · similarity 0.89)
• Off-policy estimation with adaptively collected data: the power of online learning (NeurIPS 2024 · similarity 0.88)