Research map / Label-efficient and robust learning / Recognition and long-tail
Direct Routing Gradient (DRGrad): A Personalized Information Surgery for Multi-Task Learning (MTL) Recommendations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DRGrad, a gradient-routing multi-task learning framework with a split MMoE structure, a supervised router, an updater, and a PPNet-style personalized gate, evaluated on a 15-billion-sample industrial recommender dataset, UCI Census-Income, and a synthetic dataset.
Problem
Negative transfer and the seesaw phenomenon in multi-task learning for industrial recommender systems, where task gradients conflict and noisy auxiliary tasks hurt primary tasks like click and dwell time, while existing methods add computation or ignore task importance and personalization.
Approach
Split the primary task into a dedicated tower and a shared-layer tower, use a router that routes cosine-similarity-based gradient corrections from auxiliary tasks into each part, an updater that adaptively weights the two parts via softmax over accumulated gradient norms, and a PPNet-like personalized gate on the shared layer to make gradient routing user-specific.
Key results
- DRGrad reaches 0.7651 click AUC and 0.7493 dwell time AUC on the industrial data, versus 0.7643 and 0.7483 for the best baselines (IGBv2 and AdaTask respectively), with p<=0.01 significance reported.
- On UCI Census-Income, DRGrad reaches 0.9550 Task1 AUC and 0.9949 Task2 AUC, versus best baselines of 0.9534 and 0.9948.
- Online A/B test reports 1.79% CTR gain and 0.5712% dwell time gain over MMoE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DRGrad instead of IGBv2 | MTL recommendation model · 15 Billion Samples Industry Data | Click AUC: 0.7643 -> 0.7651 | Helped |
| Use DRGrad instead of CAGrad | MTL recommendation model · 15 Billion Samples Industry Data | Dwell Time AUC: 0.7485 -> 0.7493 | Helped |
| Use DRGrad instead of Nash-MTL | MTL model · UCI Census-Income Data | Task1 AUC: 0.9534 -> 0.9550 | Helped |
| Use DRGrad instead of AdaTask | MTL model · UCI Census-Income Data | Task2 AUC: 0.9947 -> 0.9949 | Helped |
| Add the split structure to MMoE (ablation) | MTL recommendation model · 15 Billion Samples Industry Data | Click AUC: 0.7624 -> 0.7626 | Helped |
| Add the split structure to MMoE (ablation) | MTL recommendation model · 15 Billion Samples Industry Data | Dwell Time AUC: 0.7477 -> 0.7481 | Helped |
| Add the router network to Split-MMoE (ablation) | MTL recommendation model · 15 Billion Samples Industry Data | Click AUC: 0.7626 -> 0.7641 | Helped |
| Add the router network to Split-MMoE (ablation) | MTL recommendation model · 15 Billion Samples Industry Data | Dwell Time AUC: 0.7481 -> 0.7492 | Helped |
| Add PPNet to DRGrad (ablation) | DRGrad · 15 Billion Samples Industry Data | Click AUC: 0.7645 -> 0.7651 | Helped |
| Add PPNet to DRGrad (ablation) | DRGrad · 15 Billion Samples Industry Data | Dwell Time AUC: 0.7491 -> 0.7493 | Helped |
Surprising findings
- Plain Split-MMoE adds almost no AUC over MMoE (0.7626 vs 0.7624 click), so most of the gain comes from the router and personalization components.
Limitations
- Primary evaluation is on a proprietary industrial dataset that cannot be released, limiting reproducibility.
- Gains on the public Census-Income task 2 and the industrial dataset are small in absolute AUC (about 0.0001 to 0.0028).
- The convergence argument is a bounded-scale sketch rather than a full proof, and the paper does not state the hyperparameters beta and gamma used.
Experimental setup
Datasets: 15 billion sample industrial recommendation dataset UCI Census-Income synthetic dataset
Benchmarks: 15 Billion Samples Industry Data UCI Census-Income synthetic dataset
Metrics: AUC gain training time latency
Baselines: MMoE Split-MMoE SNR PLE AC-MMoE PCGrad CAGrad Nash-MTL AdaTask IGBv2
Builds on: MMoE PCGrad PPNet SNR CAGrad
Closest papers by meaning
- STEM: Unleashing the Power of Embeddings for Multi-Task Recommendation (AAAI 2024 · similarity 0.85)
- Bagging-Expert Network for Multi-Task Learning: A Depolarization Solution in Multi-Gate Mixture-of-Experts (AAAI 2025 · similarity 0.84)
- RecFlow: An Industrial Full Flow Recommendation Dataset (ICLR 2025 · similarity 0.84)
- Towards Consistent Multi-Task Learning: Unlocking the Potential of Task-Specific Parameters (CVPR 2025 · similarity 0.83)
- Preference Diffusion for Recommendation (ICLR 2025 · similarity 0.82)
- Think before Recommendation: Autonomous Reasoning-enhanced Recommender (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.