Research map / Probabilistic and scientific modeling / Molecules and proteins
Addressing Cold-Start Problem in Click-Through Rate Prediction via Supervised Diffusion Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CSDM, a non-Markovian supervised diffusion model that transitions pre-trained cold item ID embeddings toward side-information embeddings to generate warmed-up embeddings for CTR cold-start.
Problem
Cold-start items in Embedding & MLP CTR prediction have no or few user interactions, so their ID embeddings are poorly learned and CTR predictions for them are weak.
Approach
Defines a non-Markovian diffusion process between item ID embeddings and side-information hidden states (z_T = h + noise), trained jointly with the CTR BCE loss and a DDPM-style variational loss, enabling sub-sequence sampling for faster training; generated warm-up embeddings replace the original ID embeddings so inference cost is unchanged.
Key results
- On Taobao AD, CSDM(DeepFM) reaches AUC 0.6004 in the cold phase (RelaImpr 4.80%) vs CVAR 4.17%, and 18.45% RelaImpr in Warm-a vs CVAR 9.64%.
- On MovieLens-1M cold phase, CSDM(DeepFM) achieves AUC 0.7443 (RelaImpr 5.62%) vs DropoutNet 4.19% and CVAR 1.55%.
- On CIKM 2019, CSDM(DeepFM) improves over DeepFM by 1.77% in the cold phase and 1.46% in Warm-c, averaged over 10 runs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CSDM instead of CVAR | DeepFM · MovieLens-1M Cold, Warm-a, Warm-b, Warm-c | AUC: 0.7349 -> 0.7443; 0.7910 -> 0.7982; 0.8009 -> 0.8058; 0.8044 -> 0.8089 | Helped |
| Use CSDM instead of CVAR | DeepFM · Taobao AD Cold, Warm-a, Warm-b, Warm-c | AUC: 0.5998 -> 0.6004; 0.6194 -> 0.6290; 0.6295 -> 0.6324; 0.6370 -> 0.6382 | Helped |
| Use CSDM instead of CVAR | DeepFM · CIKM 2019 Cold, Warm-a, Warm-b, Warm-c | AUC: 0.7405 -> 0.7418; 0.7588 -> 0.7624; 0.7649 -> 0.7686; 0.7687 -> 0.7710 | Helped |
| Add dropout to the diffusion model (ablation) | CSDM with DeepFM · MovieLens-1M Cold, Warm-a, Warm-b, Warm-c | AUC: 0.7456 vs 0.7456; 0.7980 vs 0.7980; 0.8059 vs 0.8059; 0.8091 vs 0.8090 | No clear effect |
| Add dropout to the diffusion model (ablation) | CSDM with DeepFM · Taobao AD Cold, Warm-a, Warm-b, Warm-c | AUC: 0.5859 -> 0.6002; 0.6306 -> 0.6296; 0.6329 -> 0.6324; 0.6379 -> 0.6383 | Mixed |
| Add dropout to the diffusion model (ablation) | CSDM with DeepFM · CIKM 2019 Cold, Warm-a, Warm-b, Warm-c | AUC: 0.7402 -> 0.7418; 0.7587 -> 0.7622; 0.7677 -> 0.7687; 0.7705 -> 0.7711 | Helped |
| Add dropout to the diffusion model (ablation) | CSDM with DCN · MovieLens-1M Cold, Warm-a, Warm-b, Warm-c | AUC: 0.7360 -> 0.7362; 0.7971 vs 0.7971; 0.8020 vs 0.8020; 0.8057 -> 0.8059 | Mixed |
| Add dropout to the diffusion model (ablation) | CSDM with DCN · Taobao AD Cold, Warm-a, Warm-b, Warm-c | AUC: 0.6103 -> 0.6097; 0.6361 -> 0.6352; 0.6382 -> 0.6387; 0.6428 -> 0.6436 | Mixed |
| Add dropout to the diffusion model (ablation) | CSDM with DCN · CIKM 2019 Cold, Warm-a, Warm-b, Warm-c | AUC: 0.7425 vs 0.7425; 0.7541 -> 0.7601; 0.7629 -> 0.7657; 0.7681 -> 0.7706 | Mixed |
| Increase the loss weight ρ (ablation) | CSDM with DeepFM · Taobao AD Cold | AUC | Helped |
| Increase the loss weight ρ excessively (ablation) | CSDM with DeepFM · Taobao AD warm-up stages | AUC | Hurt |
Surprising findings
- VELF gives negative RelaImpr in some warm phases (e.g., -7.95% on MovieLens-1M Warm-b and -13.25% on CIKM Warm-c), and the dropout ablation shows dropout is not uniformly beneficial (w/o dropout is better on Taobao cold phase).
Limitations
- Training time is higher than baselines due to the diffusion process.
- Effect of dropout in the diffusion model is dataset-dependent (minimal on MovieLens-1M).
- Gains over baselines in the warm-up stage are modest on some backbones and datasets (e.g., negative RelaImpr for some baselines, small gains on CIKM).
Experimental setup
Datasets: MovieLens-1M Taobao Display Ad Click CIKM 2019 EComm AI
Benchmarks: MovieLens-1M Taobao Display Ad Click CIKM 2019 EComm AI
Metrics: AUC RelaImpr
Baselines: DeepFM Wide & Deep DCN DropoutNet MWUF Meta-E VELF CVAR
Builds on: DDPM DDIM CVAR DeepFM
Closest papers by meaning
- From Feature Interaction to Feature Generation: A Generative Paradigm of CTR Prediction Models (ICML 2025 · similarity 0.87)
- Long-Sequence Recommendation Models Need Decoupled Embeddings (ICLR 2025 · similarity 0.87)
- AT4CTR: Auxiliary Match Tasks for Enhancing Click-Through Rate Prediction (AAAI 2024 · similarity 0.87)
- Enhancing Diffusion Model with Auxiliary Information Mining-Exploration and Efficient Sampling Mechanism for Sequential Recommendation (AAAI 2025 · similarity 0.87)
- Fading to Grow: Growing Preference Ratios via Preference Fading Discrete Diffusion for Recommendation (NeurIPS 2025 · similarity 0.86)
- Learning Multiple User Distributions for Recommendation via Guided Conditional Diffusion (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.