Research map / Reinforcement learning / Policy gradient and actor-critic
High-Confidence Off-Policy (or Counterfactual) Variance Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Estimate the squared-mean term using two independent trajectory splits (double sampling) | Off-policy variance estimation · Off-policy return variance | unbiasedness | Helped |
| Use full-trajectory importance sampling for the sample variance of importance-weighted returns | Naive off-policy variance estimator · Off-policy return variance | unbiasedness | Hurt |
| Weight squared deviations by importance ratios in a plug-in variance estimator | Naive off-policy variance estimator · Off-policy return variance | unbiasedness | Hurt |
| Replace full-trajectory importance sampling with per-decision and coupled-decision importance sampling | Off-policy variance estimator · Off-policy return variance | estimator variance | Helped |
| Use interval propagation to combine confidence bounds for the second moment and squared mean | High-confidence off-policy variance estimation · Off-policy return variance | confidence-interval coverage | Helped |
| Center importance-weighted outcomes with a control variate to switch tail behavior | Concentration-based variance confidence bounds · Off-policy return variance | confidence-interval width | Helped |
| Use bootstrap confidence intervals instead of concentration-inequality intervals | High-confidence off-policy variance estimation · Off-policy return variance | confidence-interval width | Helped |
| Use bootstrap confidence intervals instead of guaranteed-coverage concentration intervals | High-confidence off-policy variance estimation · Off-policy return variance | coverage guarantee: O(n^{-1/2}) | Hurt |
Closest papers by meaning
- Universal Off-Policy Evaluation (NeurIPS 2021 · similarity 0.90)
- Towards Assessing and Benchmarking Risk-Return Tradeoff of Off-Policy Evaluation (ICLR 2024 · similarity 0.85)
- Efficient and Sharp Off-Policy Evaluation in Robust Markov Decision Processes (NeurIPS 2024 · similarity 0.84)
- Risk Estimation in a Markov Cost Process: Lower and Upper Bounds (ICML 2024 · similarity 0.84)
- On the Curses of Future and History in Future-dependent Value Functions for Off-policy Evaluation (NeurIPS 2024 · similarity 0.83)
- Log-Sum-Exponential Estimator for Off-Policy Evaluation and Learning (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.