Research map / Reinforcement learning / Policy gradient and actor-critic
Beyond IPS: Reliable Counterfactual Evaluation in Multi-Stage Ad Systems without Logged Propensities
Mohsen Malmir, Mohamed A Radwan, houssam nassif, Murat Bayir
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a counterfactual evaluation approach for offline assessment of ad ranking policies in multi-stage systems that works without logged propensities, addressing bias of Direct Method under distribution shift.
Problem
Offline evaluation of ad ranking policies in multi-stage delivery systems, where action propensities are often unavailable so IPS cannot be used, and Direct Method reward models can be biased when candidate policies induce impression distributions different from the logging policy.
Approach
Moves beyond IPS and plain Direct Method for multi-stage ad systems by estimating counterfactual value without logged propensities while correcting for impression distribution shift; abstract only describes the problem motivation, so specific mechanism is not stated.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| reweight reward-model training toward the mixture of candidate-policy distributions (Weighted DM vs Unweighted DM) (ablation) | Direct Method reward model · Ads simulator, Low ESS, Top-1 accuracy | Top-1 accuracy: 0.67 -> 0.81 | Improved |
| mixture weighting at low overlap (ablation) | Direct Method reward model · Ads simulator, Low ESS, Value MAE% | Value MAE%: 7.20 -> 2.65 | Improved |
| mixture weighting at high overlap (ablation) | Direct Method reward model · Ads simulator, High ESS, Value MAE% | Value MAE%: 1.59 -> 1.86 | Hurt |
| use one shared mixture-weighted reward model instead of per-policy weighted models (ablation) | Weighted DM · Ads simulator, Low ESS, Top-1 accuracy | Top-1 accuracy: 0.75 -> 0.81 | Improved |
| use shared mixture-weighted model instead of per-policy models (ablation) | Weighted DM · Production CTR ranking | Top-1 accuracy: 0.24 -> 0.40 | Improved |
| add mixture weighting to Direct Method (ablation) | Direct Method · Production CTR ranking | Top-1 accuracy: 0.36 -> 0.40 | Improved |
| add doubly robust residual correction to mixture-weighted DM (ablation) | Weighted DM · Ads simulator, Low ESS, Top-1 accuracy | Top-1 accuracy: 0.81 -> 0.90 | Improved |
| use mixture-weighted reward model as DR control variate instead of unweighted (ablation) | Doubly robust estimator · Ads simulator, Low ESS, Top-1 accuracy | Top-1 accuracy: 0.82 -> 0.90 | Improved |
| use mixture-weighted reward model as DR control variate (ablation) | Doubly robust estimator · Ads simulator, High ESS, Top-1 accuracy | Top-1 accuracy: 1.00 -> 0.99 | No clear effect |
| use mixture-weighted reward model as DR control variate (ablation) | Doubly robust estimator · Production CTR ranking | Top-1 accuracy / Value MAE%: Acc 0.38 -> 0.44, MAE% 1.90 -> 2.04 | Mixed |
| use classifier-estimated IPS weights instead of reward-model estimators | IPS · Production CTR ranking | Top-1 regret: IPS 0.00807 vs Weighted DM 0.00244 | Hurt |
| clip density-ratio weights at c=1 (removes all up-weighting) (ablation) | mixture weights · Ads simulator clipping sweep | Value MAE% | Hurt |
Experimental setup
Baselines: inverse propensity scoring Direct Method
Builds on: inverse propensity scoring Direct Method
Closest papers by meaning
- Doubly-Robust Estimation of Counterfactual Policy Mean Embeddings (NeurIPS 2025 · similarity 0.84)
- Beyond Last-Click: An Optimal Mechanism for Ad Attribution (NeurIPS 2025 · similarity 0.83)
- Parallel Ranking of Ads and Creatives in Real-Time Advertising Systems (AAAI 2024 · similarity 0.83)
- Online Conversion Rate Prediction via Multi-Interval Screening and Synthesizing under Delayed Feedback (AAAI 2024 · similarity 0.82)
- Percentile Risk-Constrained Budget Pacing for Guaranteed Display Advertising in Online Optimization (AAAI 2024 · similarity 0.82)
- Counterfactual Online Conformal Prediction Under Adaptive Logging (NeurIPS 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
