Research map / Learning theory and algorithms / Combinatorial and search optimization
Selective Task Group Updates for Multi-Task Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes adaptively grouping tasks and updating each group sequentially within a shared multi-task network, tracked via a proximal inter-task affinity, to mitigate negative transfer and improve task-specific learning.
Problem
Negative transfer in multi-task learning: existing gradient-manipulation and loss-weighting methods focus on shared parameters and ignore task-specific parameters, limiting their ability to mitigate task interference.
Approach
Dynamically partition tasks into groups updated sequentially within each batch, guided by a proximal inter-task affinity that is measured during optimization and accounts for both shared and task-specific parameter updates; theory links sequential group updates to better task-specific learning.
Key results
- On Taskonomy with ViT-L (11 tasks), the method reaches Δm = -1.42 versus -6.22 for the best baseline Aligned-MTL and -8.65 for GD.
- On NYUD-v2, the method achieves Δm = -2.58 versus -3.14 for Nash-MTL and -3.31 for IMTL.
- On PASCAL-Context, Δm = -6.24 versus -7.07 for IMTL and -7.20 for Aligned-MTL; the average number of groups on Taskonomy is about 1.8 with ViT-L versus 11 tasks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use selective task-group updates | Multi-task optimization · NYUD-v2 | Δm (%): -3.14\,\tiny$\pm$0.115 -> -2.58\,\tiny$\pm$0.205 | Helped |
| use selective task-group updates | Multi-task optimization · PASCAL-Context | Δm (%): -7.20\,\tiny$\pm$0.075 -> -6.24\,\tiny$\pm$0.192 | Helped |
| use selective task-group updates | Multi-task optimization · Taskonomy | Δm: -6.22\,\tiny$\pm$0.285 -> -1.42\,\tiny$\pm$ 0.208 | Helped |
| update task groups in forward order (ablation) | Selective task-group updates · Taskonomy | Δm: -1.42\,\tiny$\pm$0.208 -> -1.41\,\tiny$\pm$0.211 | No clear effect |
| update task groups in backward order (ablation) | Selective task-group updates · Taskonomy | Δm: -1.42\,\tiny$\pm$0.208 -> -1.44\,\tiny$\pm$0.209 | No clear effect |
| update all tasks jointly instead of sequentially grouped (ablation) | Selective task-group updates · Taskonomy | Δm: -1.42\,\tiny$\pm$0.208 -> -8.65\,\tiny$\pm$0.229 | Hurt |
| place each task in a separate group (ablation) | Selective task-group updates · Taskonomy | Δm: -1.42\,\tiny$\pm$0.208 -> -2.48\,\tiny$\pm$0.651 | Hurt |
| vary the affinity decay rate β (ablation) | Selective task-group updates · Multi-task benchmarks | multi-task performance | No clear effect |
Surprising findings
- Sequential group updates outperform joint gradient descent by a large margin on Taskonomy (-1.42 vs -8.65), and forward and backward group orders give similar results.
- Nash-MTL and MGDA fail to converge on Taskonomy with many tasks.
Limitations
- Requires multiple forward and backward passes per batch (M groups), increasing compute relative to loss-based methods.
- Theoretical analysis relies on assumptions such as a correlation between shared and task-specific learning and negligible affinity change within a step.
- Performance gains on some individual tasks (e.g., edge detection, saliency) are smaller than those of specialized baselines.
Experimental setup
Datasets: NYUD-v2 PASCAL-Context Taskonomy
Benchmarks: NYUD-v2 PASCAL-Context Taskonomy
Metrics: delta_m (average relative multi-task performance) mIoU RMSE L1 distance odsF maxF
Baselines: GD GradDrop MGDA UW DTP DWA PCGrad CAGrad IMTL Aligned-MTL Nash-MTL FAMO
Builds on: inter-task affinity GradNorm-style gradient balancing MGDA PCGrad Aligned-MTL
Closest papers by meaning
- Quantifying Task Priority for Multi-Task Optimization (CVPR 2024 · similarity 0.90)
- Towards Consistent Multi-Task Learning: Unlocking the Potential of Task-Specific Parameters (CVPR 2025 · similarity 0.90)
- Towards Principled Task Grouping for Multi-Task Learning (NeurIPS 2026 · similarity 0.89)
- Revisiting Fairness in Multitask Learning: A Performance-Driven Approach for Variance Reduction (CVPR 2025 · similarity 0.87)
- DMTG: One-Shot Differentiable Multi-Task Grouping (ICML 2024 · similarity 0.87)
- Fair Resource Allocation in Multi-Task Learning (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.