Research map / Learning theory and algorithms / Fairness
Fair Resource Allocation in Multi-Task Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Casts multi-task learning gradient conflict as α-fair utility maximization over task loss-decrease rates, yielding the FairGrad update, plus an α-fair loss transformation that boosts existing MTL methods.
Problem
Gradient conflicts in multi-task learning cause the update to be dominated by large gradients, hurting fair optimization of some tasks and overall MTL performance.
Approach
Model the common MTL update direction as an α-fair utility maximization (loss-decrease rates as utilities borrowed from network resource allocation), solve for task weights via a nonlinear least-squares problem, and separately apply an α-fair transformation to task losses to improve existing MTL methods.
Key results
- On Cityscapes (2-task), FairGrad achieves MR 1.50 and Δm% 5.18, versus Nash-MTL MR 3.25 and Δm% 6.82.
- On NYU-v2 (3-task), FairGrad achieves Δm% -4.66 and MR 2.67, the best among compared methods.
- On MT10 RL benchmark, FairGrad reaches 0.84 success rate, below Nash-MTL's reported 0.91 (original paper) but above several baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FairGrad instead of the best reported baseline | MTL methods · CelebA (40-task) | Δm% | Helped |
| use FairGrad instead of FAMO | MTL methods · QM9 (11-task) | Δm%: 58.5 -> 57.9 | Helped |
| use FairGrad instead of Nash-MTL | MTL methods · NYU-v2 (3-task) | MR: 3.11 -> 2.67 | Helped |
| use FairGrad instead of Nash-MTL | MTL methods · Cityscapes (2-task) | MR: 3.25 -> 1.50 | Helped |
| use FairGrad instead of Nash-MTL (reproduced) | SAC on MT10 · MT10 | success rate: 0.80 ± 0.13 -> 0.84 ± 0.07 | Helped |
| set α=2 rather than α=1 (ablation) | FairGrad · NYU-v2 (3-task) | Δm%: -2.79 -> -4.96 | Helped |
| vary α among 1, 2, 5, and 10 (ablation) | FairGrad · Cityscapes (2-task) | Δm%: 6.73; 3.90; 6.87; 10.54 | Mixed |
| add α-fair loss transformation (α=0.5) (ablation) | LS, RLW, DWA, UW, MGDA, PCGrad, and CAGrad · NYU-v2 (3-task) | Δm%: 5.59 -> 2.85; 7.78 -> 5.00; 3.57 -> 2.65; 4.05 -> 3.56; 1.38 -> -3.26; 3.97 -> 1.23; 0.20 -> -2.91 | Helped |
| add α-fair loss transformation (α=0.5) (ablation) | LS, RLW, DWA, MGDA, PCGrad, and CAGrad · Cityscapes (2-task) | Δm%: 22.60 -> 10.86; 24.38 -> 11.64; 21.45 -> 10.74; 44.14 -> 9.91; 18.29 -> 10.71; 11.64 -> 6.23 | Helped |
| add α-fair loss transformation (α=0.5) (ablation) | FairGrad · Cityscapes (2-task) | Δm%: 5.18 -> 4.92 | Helped |
| replace Nash-MTL's original subproblem solver with the authors' approach (ablation) | Nash-MTL · Cityscapes (2-task) | Δm%: 6.82 -> 6.28 | Helped |
Surprising findings
- Nash-MTL could not be reproduced exactly; reproduced result (0.80) was lower than reported (0.91).
- Applying the alpha-fair loss transformation to FairGrad gave only marginal improvement, unlike other methods which improved substantially.
Limitations
- Convergence theorem relies on linear independence of task gradients assumption outside Pareto stationary points.
- RL setting requires approximating the least-squares subproblem with SGD for time reasons.
- Best alpha requires tuning via search; MR and Δm% can disagree.
Experimental setup
Datasets: CelebA QM9 NYU-v2 Cityscapes MT10 (MetaWorld)
Benchmarks: CelebA QM9 NYU-v2 Cityscapes MT10
Metrics: Delta m% mean rank success rate
Baselines: Linear Scalarization Scale-Invariant RLW DWA UW MGDA PCGrad CAGrad IMTL-G Nash-MTL FAMO GradDrop
Builds on: Nash-MTL MGDA PCGrad CAGrad FAMO
Closest papers by meaning
- Revisiting Fairness in Multitask Learning: A Performance-Driven Approach for Variance Reduction (CVPR 2025 · similarity 0.91)
- FairMT: Fairness for Heterogeneous Multi-Task Learning (NeurIPS 2026 · similarity 0.90)
- Towards Consistent Multi-Task Learning: Unlocking the Potential of Task-Specific Parameters (CVPR 2025 · similarity 0.88)
- Bayesian Uncertainty for Gradient Aggregation in Multi-Task Learning (ICML 2024 · similarity 0.87)
- Delve into the Applicability of Advanced Optimizers for Multi-Task Learning (NeurIPS 2026 · similarity 0.87)
- Selective Task Group Updates for Multi-Task Optimization (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.