Research map / Label-efficient and robust learning / Recognition and long-tail
Generalize for Future: Slow and Fast Trajectory Learning for CTR Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SFTL, a CTR-model training framework with a slow learner (weight copy per temporal domain), a fast learner (per-iteration EMA of working weights), and a pairwise rank-based trajectory loss to generalize to future temporal domains.
Problem
CTR prediction models trained on past data with i.i.d. assumptions suffer from temporal distribution shift, leading to poor generalization to future time domains under online fine-tuning and one-pass training.
Approach
Three learners (working, slow, fast) interact: the slow learner copies working weights at each domain boundary, the fast learner tracks an EMA of working weights each iteration, and a pairwise ranking-style trajectory loss pushes the working learner to score better than these trajectory learners; inference uses the fast learner.
Key results
- On the standard supervised setting, SFTL reaches AUC 0.7833 on CIKM2019 (+1.67% over DCN-Mix) and 0.7561 on Avazu (+0.56%).
- In one-pass continual learning, SFTL achieves AUC 0.6320 on Taobao (+1.14% over incremental fine-tuning) and 0.7633 on CIKM2019 (+1.30%).
- Training is about 2.1x the cost of ERM per domain, while serving cost matches the baseline (1.0x).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SFTL | DCN-Mix · Avazu final day, supervised | AUC: 0.7517(0.0004) -> 0.7561(0.0006) | Helped |
| add SFTL | DCN-Mix · Taobao final day, supervised | AUC: 0.6378(0.0005) -> 0.6388(0.0003) | Helped |
| add SFTL | DCN-Mix · CIKM2019 final day, supervised | AUC: 0.7681(0.0003) -> 0.7833(0.0005) | Helped |
| add SFTL | DCN-Mix · Avazu final day, one-pass continual learning | AUC: 0.7468(0.0016) -> 0.7510(0.0005) | Helped |
| add SFTL | DCN-Mix · Taobao final day, one-pass continual learning | AUC: 0.6241(0.0013) -> 0.6320(0.0002) | Helped |
| add SFTL | DCN-Mix · CIKM2019 final day, one-pass continual learning | AUC: 0.7517(0.0001) -> 0.7633(0.0004) | Helped |
| add the slow trajectory learner with KL-divergence loss (ablation) | DCN-Mix continual learning · Taobao final day, one-pass continual learning | AUC: 0.6206(0.0005) -> 0.6225(0.0008) | Helped |
| use trajectory ranking loss with the slow trajectory learner instead of KL-divergence loss (ablation) | DCN-Mix continual learning · Taobao final day, one-pass continual learning | AUC: 0.6225(0.0008) -> 0.6310(0.0003) | Helped |
| add the fast trajectory learner with KL-divergence loss (ablation) | DCN-Mix continual learning · Taobao final day, one-pass continual learning | AUC: 0.6206(0.0005) -> 0.6243(0.0005) | Helped |
| use trajectory ranking loss with the fast trajectory learner instead of KL-divergence loss (ablation) | DCN-Mix continual learning · Taobao final day, one-pass continual learning | AUC: 0.6243(0.0005) -> 0.6317(0.0002) | Helped |
Surprising findings
- Flatness-aware methods (SAM, SWA) outperform drift-aware methods on most datasets, and drift-aware methods (CIDA, GI) only help on Avazu.
- Dynamic-weight architectures (SML, ASMG) underperform a simple incremental fine-tuning baseline in the one-pass setting.
Limitations
- Theoretical bound (Theorem 1) is acknowledged as possibly loose and unstable for real-world use.
- Gains on Taobao under supervised setting are small (+0.11%) and some baselines are close.
Experimental setup
Datasets: Avazu Taobao CIKM2019
Benchmarks: Avazu Taobao CIKM2019
Metrics: AUC relative improvement (Imp%) training speed
Baselines: WideDeep DeepFM DCN PNN xDeepFM AutoInt++ DCN-Mix SAM SWA CIDA GI ADER++
Builds on: DCN-Mix SAM SWA ADER CIDA
Closest papers by meaning
- Improving Transferability for Cross-Domain Trajectory Prediction via Neural Stochastic Differential Equation (AAAI 2024 · similarity 0.84)
- T4P: Test-Time Training of Trajectory Prediction via Masked Autoencoder and Actor-specific Token Memory (CVPR 2024 · similarity 0.84)
- From Feature Interaction to Feature Generation: A Generative Paradigm of CTR Prediction Models (ICML 2025 · similarity 0.84)
- Towards Stable and Storage-efficient Dataset Distillation: Matching Convexified Trajectory (CVPR 2025 · similarity 0.83)
- Addressing Cold-Start Problem in Click-Through Rate Prediction via Supervised Diffusion Modeling (AAAI 2025 · similarity 0.83)
- SingularTrajectory: Universal Trajectory Predictor Using Diffusion Model (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.