Research map / Reinforcement learning / Policy gradient and actor-critic
Counterfactual Learning with General Data-Generating Policies
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use APS controls to incorporate local observations near deterministic decision boundaries in addition to the A/B test sample | OPE with mixed stochastic and deterministic logging · Simulation Experiment 1: Mix of A/B Test and Deterministic Logging Policy | RMSE: .118 -> .112 | Helped |
| Use APS controls instead of the Direct Method | OPE with deterministic UCB logging · Simulation Experiment 2: Upper Confidence Bound Logging Policy | RMSE: .342 -> .055 | Helped |
| Use APS OPE rather than mean differences estimated from the full sample | OPE with mixed stochastic and deterministic logging · Simulation Experiment 1: Mix of A/B Test and Deterministic Logging Policy | RMSE: .128 -> .112 | Helped |
| Allow conditional mean reward differences to vary with context (ablation) | APS OPE · Simulation Experiment 1: Mix of A/B Test and Deterministic Policy, non-constant reward differences | RMSE: .019 -> .031 | Hurt |
| Allow conditional mean reward differences to vary with context (ablation) | APS OPE · Simulation Experiment 2: Upper Confidence Bound, non-constant reward differences | RMSE: .117 -> .119 | Hurt |
| Use bandwidth δ = 0.1 instead of δ = 1 (ablation) | APS OPE · Simulation Experiment 1: Mix of A/B Test and Deterministic Logging Policy | RMSE: .112 -> .115 | Hurt |
| Use bandwidth δ = 2.5 instead of δ = 1 (ablation) | APS OPE · Simulation Experiment 1: Mix of A/B Test and Deterministic Logging Policy | RMSE: .112 -> .113 | No clear effect |
| Use bandwidth δ = 0.1 instead of δ = 1 (ablation) | APS OPE · Simulation Experiment 2: Upper Confidence Bound Logging Policy | RMSE: .055 -> .058 | Hurt |
| Use bandwidth δ = 2.5 instead of δ = 1 (ablation) | APS OPE · Simulation Experiment 2: Upper Confidence Bound Logging Policy | RMSE: .055 -> .055 | No clear effect |
| Expand coupon targeting using the APS-estimated effects | Mercari coupon targeting policy · Coupon campaign policy recommendation | Coupon Cost Effectiveness Measure: 79.57; 96.35; 134; 93.51; 92.07 | Helped |
Closest papers by meaning
- Efficient Counterfactual Learning from Bandit Feedback (AAAI 2019 · similarity 0.92)
- Cross-Domain Off-Policy Evaluation and Learning for Contextual Bandits (ICLR 2025 · similarity 0.89)
- A General Framework for Off-Policy Learning with Partially-Observed Reward (ICLR 2025 · similarity 0.88)
- Budgeted Multi-Source Counterfactual Annotation for Off-Policy Evaluation (NeurIPS 2026 · similarity 0.88)
- Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learning (NeurIPS 2024 · similarity 0.87)
- Log-Sum-Exponential Estimator for Off-Policy Evaluation and Learning (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.