Research map / Learning theory and algorithms / Combinatorial and search optimization

Towards Principled Task Grouping for Multi-Task Learning

NeurIPS 2026 · Learning theory and algorithms › Combinatorial and search optimization · approach: Multi-objective optimization · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes an assumption-free transfer gain measure for task grouping in MTL, plus a mixed-integer programming formulation that handles budget and group-size constraints, with sampling and lazy collection for efficiency.

Problem

Task grouping in multi-task learning must separate positive from negative transfer, but prior affinity measures (TAG) rely on restrictive assumptions (convexity, smoothness) and grouping is solved by enumeration/branch-and-bound that is costly and hard to extend with constraints.

Approach

Defines a lookahead transfer gain from one-step loss reduction when training a task with a source task (no convexity/smoothness assumptions), approximates group gains by averages of pairwise gains, and solves grouping as a quadratic/mixed-integer program with knapsack-style budget and group-size constraints solved by Gurobi.

Key results

• On Taskonomy with 2 groups, Ours achieves loss reduction 0.499 (second only to Optimal); HOA reaches 0.608 but is unstable across splits.
• On CelebA, Ours reaches total error 49.534 at 2 splits vs 49.583 for Optimal (TAG's optimal grouping), beating other non-optimal grouping methods.
• Lazy collection with 10-50 step intervals reportedly gives 10-15x speedups while maintaining or improving performance.

Surprising findings

• On COP, single-task learning outperforms all MTL methods in total gap, and less frequent transfer-gain collection (up to 50 steps) on CelebA improved grouping quality.
• Optimal groupings on COP do not pair tasks of the same problem type, contrary to the intuitive pairing the proposed method finds.

Limitations

• Scalability to an extremely large number of tasks is not established.
• The MIP framework currently only handles knapsack-style constraints.
• The bound in the proposition relies on Lipschitz gradient bounds and a positive loss lower bound, and the group-to-average approximation is only approximate.

Experimental setup

Datasets: Taskonomy CelebA TSP20 TSP50 CVRP20 CVRP50 OP20 OP50 ETTm1

Benchmarks: Taskonomy CelebA combinatorial optimization benchmarks (TSP, CVRP, OP) ETTm1

Metrics: loss reduction vs STL total classification error total optimality gap (%) total MAE

Baselines: STL Naive-MTL Bandit-MTL PCGrad Nash-MTL Uncertainty Weighting TAG HOA MTG Random grouping Optimal grouping

Builds on: TAG high-order approximation POMO Autoformer

Closest papers by meaning

• DMTG: One-Shot Differentiable Multi-Task Grouping (ICML 2024 · similarity 0.90)
• Selective Task Group Updates for Multi-Task Optimization (ICLR 2025 · similarity 0.89)
• Towards Consistent Multi-Task Learning: Unlocking the Potential of Task-Specific Parameters (CVPR 2025 · similarity 0.88)
• Task Groupings Regularization: Data-Free Meta-Learning with Heterogeneous Pre-trained Models (ICML 2024 · similarity 0.87)
• Quantifying Task Priority for Multi-Task Optimization (CVPR 2024 · similarity 0.87)
• Task-Aware Clustering for Prompting Vision-Language Models (CVPR 2025 · similarity 0.87)