Research map / Architectures and efficiency / RNNs and normalization
AdaTask: A Task-Aware Adaptive Learning Rate Approach to Multi-Task Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| separate each task's accumulative gradients and adaptive learning rate for shared parameters | SharedBottom MTL on Synthetic · Synthetic | Task A RMSE: 0.2172 -> 0.0704 | Helped |
| separate each task's accumulative gradients and adaptive learning rate for shared parameters | SharedBottom MTL on Synthetic · Synthetic | Task B RMSE: 0.0113 -> 0.0417 | Hurt |
| separate each task's accumulative gradients and adaptive learning rate for shared parameters | MTAN on CityScapes · CityScapes | Abs Err: 0.0152 -> 0.0128 | Helped |
| separate each task's accumulative gradients and adaptive learning rate for shared parameters | SharedBottom MTL · TikTok | Task A AUC: 0.9173 -> 0.9312 | Helped |
| separate each task's accumulative gradients and adaptive learning rate for shared parameters | SharedBottom MTL · TikTok | Task B AUC: 0.7437 -> 0.7399 | Hurt |
| separate each task's accumulative gradients and adaptive learning rate for shared parameters | SharedBottom MTL · WeChat | Average AUC: 0.9338 -> 0.9359 | Helped |
| separate each task's accumulative gradients and adaptive learning rate for shared parameters | SharedBottom MTL · WeChat | Weighted AUC: 0.9336 -> 0.9311 | Hurt |
| combine AdaTask with PCGrad | MTAN on CityScapes · CityScapes | Δp: 9.34% -> 10.52% | Helped |
| combine AdaTask with CAGrad | MTAN on CityScapes · CityScapes | Δp: 9.34% -> 10.59% | Helped |
| use layer-wise accumulators (LAdaTask) instead of parameter-wise task accumulators (ablation) | SharedBottom MTL · Synthetic | Average Task-Normalized RMSE: 0.0560 -> 0.0637 | Hurt |
| use layer-wise accumulators (LAdaTask) instead of parameter-wise task accumulators (ablation) | SharedBottom MTL · TikTok | Average AUC: 0.8356 -> 0.8321 | Hurt |
| apply AdaTask to MMoE rather than use EqualWeight on MMoE | MMoE on WeChat · WeChat | Average AUC: +1.89% | Helped |
Closest papers by meaning
- Towards Consistent Multi-Task Learning: Unlocking the Potential of Task-Specific Parameters (CVPR 2025 · similarity 0.87)
- Towards Principled Task Grouping for Multi-Task Learning (NeurIPS 2026 · similarity 0.87)
- Delve into the Applicability of Advanced Optimizers for Multi-Task Learning (NeurIPS 2026 · similarity 0.87)
- AdaMerging: Adaptive Model Merging for Multi-Task Learning (ICLR 2024 · similarity 0.86)
- TADFormer: Task-Adaptive Dynamic TransFormer for Efficient Multi-Task Learning (CVPR 2025 · similarity 0.86)
- Bayesian Uncertainty for Gradient Aggregation in Multi-Task Learning (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.