Research map / Label-efficient and robust learning / Recognition and long-tail
STEM: Unleashing the Power of Embeddings for Multi-Task Recommendation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes STEM, a shared-plus-task-specific embedding paradigm with STEM-Net (All Forward Task-specific Backward gating) to fix negative transfer in multi-task recommendation, evaluated on TikTok, QK-Video and KuaiRand1K plus an industrial A/B test.
Problem
Negative transfer in multi-task recommendation occurs even on samples with comparable positive feedback across tasks, because shared-embedding models can only encode a single user preference per item.
Approach
Adds task-specific embedding tables alongside shared ones; a gating network receives forward signals from all experts but uses stop-gradient on other tasks' task-specific experts, so each task learns its own embeddings while still transferring knowledge directly.
Key results
- STEM-Net achieves the best average AUC on all three datasets: 0.8304 on TikTok, 0.9331 on QK-Video, 0.8480 on KuaiRand1K, about 2e-3 to 4e-3 above PLE.
- On the TikTok comparable subset, STEM-Net achieves positive transfer over the single-task Like model, where MMoE and PLE do not.
- In online A/B tests on Tencent's advertising platform, STEM-Net gave 0.32%, 0.24%, 0.48% average AUC lifts and 4.2%, 3.9%, 7.1% GMV lift over the MMoE production model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use STEM-Net instead of ME-PLE | Multi-task recommendation · TikTok | Average AUC: 0.8275 -> 0.8304 | Helped |
| Use STEM-Net instead of ME-PLE | Multi-task recommendation · QK-Video | Average AUC: 0.9324 -> 0.9331 | Helped |
| Use STEM-Net instead of ME-PLE | Multi-task recommendation · KuaiRand1K | Avg. AUC: 0.8445 -> 0.8480 | Helped |
| Add task-specific embeddings for all feature fields rather than none (ablation) | STEM-Net · TikTok | AUC: 0.8261 -> 0.8304 | Helped |
| Add task-specific embeddings for user ID and item ID (ablation) | STEM-Net · TikTok | AUC: 0.8261 -> 0.8302 | Helped |
| Use task-specific embeddings for user-side fields instead of none (ablation) | STEM-Net · TikTok | AUC: 0.8261 -> 0.8301 | Helped |
| Use task-specific embeddings for item-side fields instead of none (ablation) | STEM-Net · TikTok | AUC: 0.8261 -> 0.8260 | No clear effect |
| Use task-specific embeddings for all feature fields rather than none (ablation) | STEM-Net · KuaiRand1K | AUC: 0.8382 -> 0.8480 | Helped |
| Use task-specific embeddings for user ID and item ID (ablation) | STEM-Net · KuaiRand1K | AUC: 0.8382 -> 0.8448 | Helped |
| Use task-specific embeddings for user-side fields instead of none (ablation) | STEM-Net · KuaiRand1K | AUC: 0.8382 -> 0.8437 | Helped |
| Use task-specific embeddings for item-side fields instead of none (ablation) | STEM-Net · KuaiRand1K | AUC: 0.8382 -> 0.8193 | Hurt |
| Add stop-gradient to the task-specific-input gate (ablation) | STEM-Net · TikTok | Like AUC: 0.9023 -> 0.9077 | Helped |
Surprising findings
- Negative transfer persists for existing MTL models on samples with comparable positive feedback across tasks, even though they help on overwhelming subsets.
- Simply adding per-expert embedding tables (ME-MMoE) without the stop-gradient gating gives little gain, so the gating design rather than extra parameters drives the improvement.
Limitations
- Gains on some tasks are small (e.g., KuaiRand1K tasks where STEM-Net is not best), and the parameter count increases with multiple embedding tables (up to 2.85x on TikTok).
- Evaluation limited to three public datasets and one industrial advertising deployment; the analysis of contradictory preferences uses a heuristic distance-based selection.
Experimental setup
Datasets: TikTok QK-Video KuaiRand1K
Benchmarks: TikTok QK-Video KuaiRand1K
Metrics: AUC Logloss Average AUC MTL Gain GMV lift (online A/B)
Baselines: Single-Task Shared-Bottom OMoE MMoE PLE ESMM AITM ME-MMoE ME-PLE
Builds on: MMoE PLE ESMM Shared-Bottom
Closest papers by meaning
- Direct Routing Gradient (DRGrad): A Personalized Information Surgery for Multi-Task Learning (MTL) Recommendations (AAAI 2025 · similarity 0.85)
- Tree of Preferences for Diversified Recommendation (NeurIPS 2025 · similarity 0.85)
- Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations (ICML 2024 · similarity 0.84)
- FACE: A General Framework for Mapping Collaborative Filtering Embeddings into LLM Tokens (NeurIPS 2025 · similarity 0.84)
- Bagging-Expert Network for Multi-Task Learning: A Depolarization Solution in Multi-Gate Mixture-of-Experts (AAAI 2025 · similarity 0.84)
- Objective Shaping with Hard Negatives: Windowed Partial AUC Optimization for RL-based LLM Recommenders (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.