Research map / Label-efficient and robust learning / Recognition and long-tail
From Feature Interaction to Feature Generation: A Generative Paradigm of CTR Prediction Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reformulates discriminative CTR models as a generative All-Predict-All feature generation framework (SFG) with an MLP encoder and decoder trained with a supervised click loss, improving AUC/Logloss on Criteo and Avazu and mitigating embedding collapse and redundancy.
Problem
CTR prediction models follow a discriminative paradigm that relies on interactions between raw ID embeddings, which causes embedding dimensional collapse and information redundancy.
Approach
Treat all feature embeddings as source and generate all feature embeddings as target via a field-wise MLP encoder and a projection decoder ("All-Predict-All"), trained with the supervised click label instead of a self-supervised loss, so that interactions are computed on newly generated, decorrelated embeddings rather than raw ID embeddings.
Key results
- Generative paradigm improves AUC by an average of 0.272% and reduces Logloss by 0.435% across eight CTR base models on Criteo and Avazu.
- Online A/B test on Tencent advertising platform yields 2.68% GMV lift and 2.46% CTR lift.
- Generative paradigm adds an average 3.14% training time and 1.45% GPU memory overhead.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply supervised feature generation to DCN V2 | DCN V2 · Criteo | AUC: 0.81387 -> 0.81472 | Helped |
| Apply supervised feature generation to DCN V2 | DCN V2 · Criteo | Logloss: 0.43826 -> 0.43713 | Helped |
| Apply supervised feature generation to DCN V2 | DCN V2 · Avazu | AUC: 0.79282 -> 0.79342 | Hurt |
| Apply supervised feature generation to DCN V2 | DCN V2 · Avazu | Logloss: 0.37222 -> 0.37180 | Hurt |
| Use all feature-field embeddings as the source instead of only each field's own embedding (ablation) | DCN V2 feature generation · Avazu | recommendation performance | Helped |
| Construct embeddings from all features for low-cardinality fields rather than high-cardinality fields (ablation) | DCN V2 feature generation · Avazu | recommendation performance | Helped |
| Replace the field-wise encoder MLP with a field-shared MLP (ablation) | DCN V2 feature generation · Avazu | recommendation performance | Hurt |
| Remove the encoder's nonlinear activation (ablation) | DCN V2 feature generation · Avazu | recommendation performance | Hurt |
| Add one encoder layer (ablation) | DCN V2 feature generation · Avazu | AUC: 0.7935 -> 0.7929 | Hurt |
| Replace the MLP encoder with self-attention (ablation) | DCN V2 feature generation · Avazu | recommendation performance | Mixed |
| Generate only randomly selected target fields instead of all fields (ablation) | DCN V2 feature generation · Avazu | recommendation performance | Hurt |
| Use a single learnable mask vector for masked feature modeling (ablation) | DCN V2 feature generation · Avazu | recommendation performance | Hurt |
Surprising findings
- Reformulated CrossNet V2 (generative) outperforms discriminative DCN V2 on both datasets.
- A learnable mask vector in masked feature modeling hurts performance, while a fixed zero mask works better.
Limitations
- Encoder cost grows with the square of the number of fields, motivating source restriction to low-cardinality fields for efficiency.
- Gains on DNN-based models are smaller than on explicit interaction models.
Experimental setup
Datasets: Criteo Avazu
Benchmarks: Criteo Avazu
Metrics: AUC Logloss singular value spectrum Pearson correlation training time per epoch GPU memory
Baselines: FM FmFM CrossNet V2 DeepFM xDeepFM IPNN DCN V2 Fi-GNN GFRL FRNet
Builds on: FM DeepFM DCN V2 xDeepFM FmFM
Closest papers by meaning
- Addressing Cold-Start Problem in Click-Through Rate Prediction via Supervised Diffusion Modeling (AAAI 2025 · similarity 0.87)
- AT4CTR: Auxiliary Match Tasks for Enhancing Click-Through Rate Prediction (AAAI 2024 · similarity 0.86)
- Long-Sequence Recommendation Models Need Decoupled Embeddings (ICLR 2025 · similarity 0.84)
- Generalize for Future: Slow and Fast Trajectory Learning for CTR Prediction (AAAI 2024 · similarity 0.84)
- On the Embedding Collapse when Scaling up Recommendation Models (ICML 2024 · similarity 0.83)
- Language Ranker: A Lightweight Ranking framework for LLM Decoding (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.