Research map / Architectures and efficiency / Pruning and efficient tuning

No More Tuning: Prioritized Multi-Task Learning with Lagrangian Differential Multiplier Methods

AAAI 2025 · Architectures and efficiency › Pruning and efficient tuning · approach: Parameter-efficient transfer analysis · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes No More Tuning (NMT), a step-wise constrained-optimization MTL framework that uses Lagrange multipliers to prioritize a primary task without loss-weight hyperparameter tuning, evaluated on TikTok, QK-Video and Taobao search A/B tests.

Problem

Multi-task learning with tasks of different priorities: manual loss-weight tuning grows exponentially with the number of tasks, and joint optimization can degrade high-priority task performance.

Approach

Cast prioritized MTL as sequential constrained optimization where each lower-priority task is minimized subject to not exceeding the optimal values of higher-priority tasks, solved via gradient descent on parameters and gradient ascent on Lagrange multipliers, with no extra hyperparameters.

Key results

• On TikTok, NMT raises Like AUC by +0.38% to +0.49% across Shared-Bottom, OMoE, MMoE and PLE, with Finish AUC also slightly improving.
• On QK-Video, Like AUC improves by +0.03% to +0.10% with NMT, while Click AUC shifts by at most about -0.07%.
• In Taobao A/B tests, pay + relevance with NMT gives +0.26% order volume and +0.72% relevance, whereas manual weight adjustment drops order volume by -0.35%.

Measured effects reported in the paper

Surprising findings

• Weight adjustment in the Taobao pay + relevance setup reduced order volume by -0.35%, while NMT did not reduce it.
• On QK-Video, NMT slightly lowered Click AUC for some models while raising Like AUC.

Limitations

• Strong-duality guarantee relies on assumptions (no overfitting, small parameter change) and is only approximate for non-convex models.
• Theoretical argument is partly informal; the convexity proof is approximate (O(epsilon) terms).
• Tasks are prioritized by a fixed order; the method was not compared against tuned weight grids on public datasets beyond a TikTok sweep.

Experimental setup

Datasets: TikTok QK-Video Taobao search logs

Benchmarks: TikTok QK-Video Taobao search A/B test

Metrics: AUC order volume GMV relevance

Baselines: Shared-Bottom OMoE MMoE PLE FAMO weight adjustment

Builds on: Lagrangian methods MMoE PLE FAMO

Closest papers by meaning

• Towards Consistent Multi-Task Learning: Unlocking the Potential of Task-Specific Parameters (CVPR 2025 · similarity 0.87)
• Quantifying Task Priority for Multi-Task Optimization (CVPR 2024 · similarity 0.87)
• Delve into the Applicability of Advanced Optimizers for Multi-Task Learning (NeurIPS 2026 · similarity 0.86)
• Revisiting Fairness in Multitask Learning: A Performance-Driven Approach for Variance Reduction (CVPR 2025 · similarity 0.86)
• Bayesian Uncertainty for Gradient Aggregation in Multi-Task Learning (ICML 2024 · similarity 0.85)
• Selective Task Group Updates for Multi-Task Optimization (ICLR 2025 · similarity 0.85)