Research map / Learning theory and algorithms / Combinatorial and search optimization
Do Current Multi-Task Optimization Methods in Deep Learning Even Help?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MTO algorithms (MGDA, GradNorm, PCGrad, IMTL, RLW) instead of scalarization | Transformer NMT · En->{Fr,Zh}, En->{Fr,De} | train/test loss trade-off | No clear effect |
| use MTO algorithms instead of tuned scalarization | Transformer NMT · En->{Fr,Ro} test generalization | test loss | Hurt |
| use MTO algorithms instead of scalarization | ResNet-50 segmentation+depth · CityScapes | test loss/accuracy trade-off | Hurt |
| use MGDA instead of scalarization | CelebA multitask model · CelebA test | average error: 9.23 -> 9.53 | Hurt |
| use PCGrad instead of scalarization | CelebA multitask model · CelebA test | average error: 9.23 -> 9.35 | No clear effect |
| scale model depth (3L to 12L) with scalarization instead of using per-task-gradient MTO | Transformer NMT · En->{Zh,Fr} | Pareto front: 11.52 -> 5.48 steps/sec for 12L | Helped |
| tune learning rate on a sparse grid versus varying seed | MTO evaluation · NMT | performance variance | Hurt |
Closest papers by meaning
- Delve into the Applicability of Advanced Optimizers for Multi-Task Learning (NeurIPS 2026 · similarity 0.88)
- Towards Consistent Multi-Task Learning: Unlocking the Potential of Task-Specific Parameters (CVPR 2025 · similarity 0.87)
- Joint Scheduling of Causal Prompts and Tasks for Multi-Task Learning (CVPR 2025 · similarity 0.86)
- MTLoRA: Low-Rank Adaptation Approach for Efficient Multi-Task Learning (CVPR 2024 · similarity 0.86)
- Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignment (CVPR 2025 · similarity 0.86)
- Benchmarking Optimizers for Large Language Model Pretraining (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.