Research map / Learning theory and algorithms / Combinatorial and search optimization
Quantifying Task Priority for Multi-Task Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes connection strength-based optimization for multi-task learning that learns task priority from task-specific BN-scaled connection strengths and projects gradients to preserve it, reporting gains over gradient-manipulation methods on NYUD-v2, PASCAL-Context, and Cityscapes.
Problem
Negative transfer and conflicting gradients in multi-task learning; prior gradient-alignment methods do not identify which shared parameters matter to which tasks, yielding sub-optimal Pareto solutions.
Approach
Defines task priority over shared parameters via task-specific connection strength (squared kernel weights scaled by task-specific batch-norm gamma, normalized across channels), then alternates a sequential per-task update phase with a priority-preserving gradient projection phase.
Key results
- On NYUD-v2 with HRNet-18, the method reaches +15.00% multi-task performance versus +9.53% for GD and +10.17% for Aligned-MTL.
- On PASCAL-Context with HRNet-18, it reaches +3.70% versus +1.86% for PCGrad and +1.47% for GD.
- On Cityscapes with SegNet, it achieves 74.75 mIoU and +0.0125 abs error with only about 0.05% parameter increase over the base model, averaging over 3 seeds.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use connection strength-based optimization instead of Aligned-MTL | Multi-task learning with HRNet-18 · NYUD-v2 | MTP (△m ↑ %): + 10.17 -> + 15.00 | Helped |
| Use connection strength-based optimization instead of PCGrad | Multi-task learning with HRNet-18 · PASCAL-Context | MTP (△m ↑ %): + 1.86 -> + 3.70 | Helped |
| Use connection strength-based optimization instead of MGDA with Recon | Multi-task learning with SegNet · Cityscapes | MTP (△m ↑ %): - 4.46 -> - 10.08 | Hurt |
| Use Phase 1 only rather than mixing both phases (ablation) | Connection strength-based optimization · NYUD-v2 | MTP (△m ↑): + 15.50 -> + 13.44 | Hurt |
| Use Phase 2 only rather than mixing both phases (ablation) | Connection strength-based optimization · NYUD-v2 | MTP (△m ↑): + 15.50 -> + 10.32 | Hurt |
| Apply Phase 1 and Phase 2 sequentially instead of mixing them (ablation) | Connection strength-based optimization · NYUD-v2 | MTP (△m ↑): + 15.50 -> + 13.79 | Hurt |
Surprising findings
- Using only Phase 2 (priority-conserving projection) performs about the same as previous gradient-manipulation methods, and sequential application of the two phases underperforms the mixed schedule.
Limitations
- Task priority computation is defined per convolutional layer followed by task-specific batch norm; extension to transformers or linear layers is only suggested, not tested.
- Relies on manually tuned or scheduled loss weights in main experiments and on a mixing schedule between phases that depends on epochs.
- Theoretical claim (Theorem 1) relies on small learning rate and differentiability assumptions.
Experimental setup
Datasets: NYUD-v2 PASCAL-Context Cityscapes
Benchmarks: NYUD-v2 PASCAL-Context Cityscapes
Metrics: multi-task performance (delta m %) RMSE abs rel mIoU pixel accuracy surface normal angle error and within-t-degree accuracy maxF absolute and relative depth error
Baselines: GD MGDA PCGrad CAGrad Aligned-MTL Graddrop RotoGrad Recon Cross-Stitch single-task learning
Builds on: PCGrad MGDA CAGrad Aligned-MTL MTI-Net
Closest papers by meaning
- Selective Task Group Updates for Multi-Task Optimization (ICLR 2025 · similarity 0.90)
- Towards Consistent Multi-Task Learning: Unlocking the Potential of Task-Specific Parameters (CVPR 2025 · similarity 0.88)
- No More Tuning: Prioritized Multi-Task Learning with Lagrangian Differential Multiplier Methods (AAAI 2025 · similarity 0.87)
- Towards Principled Task Grouping for Multi-Task Learning (NeurIPS 2026 · similarity 0.87)
- Hard Tasks First: Multi-Task Reinforcement Learning Through Task Scheduling (ICML 2024 · similarity 0.86)
- Fair Resource Allocation in Multi-Task Learning (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.