Research map / Reinforcement learning / Policy gradient and actor-critic
Importance Sampling Policy Evaluation with an Estimated Behavior Policy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace true behavior-policy weights with weights from a behavior policy estimated on the same evaluation data | Importance sampling variants · Gridworld | MSE | Helped |
| Estimate the behavior policy on independent training data instead of the evaluation data (ablation) | Importance sampling · Gridworld | MSE | Hurt |
| Use both independent extra data and evaluation data to estimate the behavior policy (ablation) | Importance sampling · Gridworld | MSE | Mixed |
| Estimate the behavior policy on the evaluation data only | Importance sampling · Linear Dynamical System | MSE | Helped |
| Use RIS weights instead of OIS weights | Ordinary IS and PDIS · Linear Dynamical System | MSE | Helped |
| Use RIS weights instead of OIS weights | WIS · Linear Dynamical System | MSE | No clear effect |
| Increase history length from RIS(0) to larger n (ablation) | RIS · SinglePath | MSE | Mixed |
| Use RIS(4) instead of RIS(3) (ablation) | RIS · SinglePath | MSE: more than 20 orders of magnitude below | Helped |
| Use RIS(0) instead of OIS | Importance sampling · SinglePath, low-to-medium data regime | MSE | Helped |
| Use a two-hidden-layer behavior-policy network rather than OIS | RIS · Hopper | MSE | Helped |
| Use a two-hidden-layer behavior-policy network rather than OIS | RIS · HalfCheetah | MSE | No clear effect |
| Select the behavior-policy model by minimum validation loss rather than minimum RIS MSE (ablation) | RIS neural-network model selection · Hopper | MSE: ≈25% | Hurt |
Closest papers by meaning
- Demystifying the Paradox of Importance Sampling with an Estimated History-Dependent Behavior Policy in Off-Policy Evaluation (ICML 2025 · similarity 0.93)
- Towards Optimal Off-Policy Evaluation for Reinforcement Learning with Marginalized Importance Sampling (NeurIPS 2019 · similarity 0.92)
- Data-Efficient Policy Evaluation Through Behavior Policy Search (ICML 2017 · similarity 0.91)
- Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learning (NeurIPS 2024 · similarity 0.89)
- Efficient and Sharp Off-Policy Evaluation in Robust Markov Decision Processes (NeurIPS 2024 · similarity 0.88)
- Efficient Policy Evaluation with Offline Data Informed Behavior Policy Design (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.