Research map / Label-efficient and robust learning / Recognition and long-tail
Online Conversion Rate Prediction via Multi-Interval Screening and Synthesizing under Delayed Feedback
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MISS, a multi-head screening model over multiple waiting windows plus a light-weight synthesizing model trained on an assembled pipeline and a global positive weighting, for delayed-feedback online CVR prediction.
Problem
Online conversion rate prediction under delayed feedback, where short waiting windows introduce fake negatives and long windows cause data staleness, and existing aggregation of multi-window models lacks consensus.
Approach
A shared-bottom network with output heads trained on duplicated data pipelines of different waiting windows, aggregated by a light-weight softmax-weighted synthesizing model trained on the latest converted positives plus real negatives, and a global importance-style positive weighting controlled by a single hyperparameter alpha instead of per-sample weight models.
Key results
- On Criteo, MISS reaches AUC 83.7%, NLL 83.9%, PR-AUC 78.1% in relative-improvement terms, the best among all baselines (e.g., DEFUSE 66.9% AUC, FTP 59.5%).
- On Tencent, MISS reaches AUC 86.0% vs. FTP 79.9% in relative-improvement terms, a 6.1% relative gain over the strongest baseline.
- Ablation on Criteo shows removing the duplication mechanism lowers AUC from 0.8477 to 0.8451, and removing the synthesizing model (using last head) drops AUC to 0.8390.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MISS instead of the strongest baseline, DEFUSE | Criteo Dataset · Criteo Dataset | AUC: 16.8% | Helped |
| use MISS instead of the strongest baseline, FTP | Tencent Dataset · Tencent Dataset | AUC: 6.1% | Helped |
| add the duplication mechanism to the screening heads (ablation) | MISS · Criteo | AUC: 0.8451 -> 0.8477 | Helped |
| add the synthesizing model instead of using the last head prediction (ablation) | MISS · Criteo | AUC: 0.8390 -> 0.8477 | Helped |
| use dynamic aggregation instead of averaging head predictions (ablation) | MISS · Criteo | AUC: 0.8469 -> 0.8477 | Helped |
| train the synthesizing model on the assembled pipeline instead of the maximum-window dataset (ablation) | MISS · Criteo | PR-AUC: 0.6430 -> 0.6495 | Helped |
| use head predictions alone instead of adding middle-layer results to synthesizing-model inputs (ablation) | MISS · Criteo | AUC: 0.8477 -> 0.8477 | No clear effect |
| use head predictions alone instead of adding middle-layer results to synthesizing-model inputs (ablation) | MISS · Criteo | NLL: 0.3857 -> 0.3856 | No clear effect |
Surprising findings
- Setting the positive-weight hyperparameter alpha higher than the global average CVR continued to lower NLL, which the authors attribute to excess positive weights mimicking down-weighting of negatives.
- A synthesizing model trained on the maximum-attribution-window data gave dominant weight to the longest-window head, ignoring other heads.
Limitations
- Global positive weighting uses a manually set hyperparameter alpha rather than a principled estimate of the ideal CVR.
- Evaluation is limited to two advertising datasets.
- Assumes ideal distribution approximated by importance sampling; the synthesizing model depends on the assembled pipeline design.
Experimental setup
Datasets: Criteo Conversion Logs Tencent Advertising Algorithm Competition 2017
Benchmarks: Criteo Conversion Logs Tencent Advertising Algorithm Competition 2017
Metrics: AUC NLL PR-AUC
Baselines: Vanilla FNW FNC ES-DFM DEFUSE MTDFM FTP
Builds on: ES-DFM FNW FTP DEFUSE importance sampling
Closest papers by meaning
- Capturing Delayed Feedback in Conversion Rate Prediction via Elapsed-Time Sampling (AAAI 2021 · similarity 0.93)
- Entire-Space Variational Information Exploitation for Post-Click Conversion Rate Prediction (AAAI 2025 · similarity 0.85)
- Percentile Risk-Constrained Budget Pacing for Guaranteed Display Advertising in Online Optimization (AAAI 2024 · similarity 0.81)
- Bandit and Delayed Feedback in Online Structured Prediction (NeurIPS 2025 · similarity 0.81)
- Bi-Level Decision-Focused Causal Learning for Large-Scale Marketing Optimization: Bridging Observational and Experimental Data (NeurIPS 2025 · similarity 0.80)
- Online Learning of Delayed Choices (NeurIPS 2024 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.