Research map / Learning theory and algorithms / Combinatorial and search optimization
Deconstructing Multi-Task Active Learning: The Paradox of Gradient Conflict and Orthogonal Decomposition
Juho Jung, Seongjae Kang, Jinyoung Han
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Argues that linear scalarization of heterogeneous active-learning objectives (uncertainty, diversity, gradient conflict) in multi-task active learning is fundamentally limited and proposes moving from combination to an orthogonal decomposition view.
Problem
Multi-task active learning (MTL-AL) methods typically combine heterogeneous sample-selection objectives such as uncertainty, diversity, and gradient conflict via linear scalarization, which may fail to capture Pareto-optimal trade-offs under conflicting objectives.
Approach
Replaces the entangled linear combination of selection objectives with an orthogonal decomposition formulation, shifting from combining objectives to decomposing them so that conflicting signals can be handled Pareto-optimally.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| penalize gradient conflict in the acquisition score vs ignoring it (ablation) | MTL active learning (AdamW) · COCO | MTL composite: 0.0756 -> 0.0589 | Hurt |
| embrace conflict-informed selection with PCGrad vs ignoring conflict (ablation) | ODS without Stage 2 · COCO | MTL composite: 0.0756 -> 0.0932 | Improved |
| add Stage 2 k-center coverage after informativeness filtering (ablation) | ODS · COCO | MTL composite: 0.0932 -> 0.105 | Improved |
| add CTMI to Hardness in the informativeness score (ablation) | ODS · COCO | MTL composite: 0.094 -> 0.105 | Improved |
| change combination rule of Hardness and CTMI (rank-sum, z-score, product) (ablation) | ODS · COCO | MTL composite: 0.723-0.731 | No clear effect |
| two-stage orthogonal decomposition vs linear scalarization of the same scores (ablation) | ODS · COCO seed 42 | MTL composite: +6.5% | Improved |
| use PCGrad instead of AdamW with ODS (ablation) | ODS · COCO | delta m (%): +8.2 -> +10.5 | Improved |
| ODS versus MT-BADGE under the same optimizer | ODS · COCO | delta m (%): +7.5 -> +10.5 (PCGrad) | Improved |
| entropy or CoreSet selection vs random | Entropy, CoreSet · COCO | MTL composite: -0.005, -0.002 | Hurt |
| ODS versus adapted two-stage methods (FASS, Cluster-Margin, PPAL, D2-Pruning, SES) | ODS · COCO unified protocol | best delta m (%): 4.41 vs 2.92-4.15 | No clear effect |
| ODS versus adapted two-stage methods on whole trajectory | ODS · COCO unified protocol | trajectory delta m (%): +1.03 vs -0.15 to +0.41 | Improved |
| ODS versus random sampling on other datasets | ODS · Cityscapes, NYUv2 | delta m (%): +8.8, +7.5 | Improved |
Closest papers by meaning
- Towards Consistent Multi-Task Learning: Unlocking the Potential of Task-Specific Parameters (CVPR 2025 · similarity 0.87)
- PiKE: Adaptive Data Mixing for Large-Scale Multi-Task Learning Under Low Gradient Conflicts (NeurIPS 2025 · similarity 0.87)
- Delve into the Applicability of Advanced Optimizers for Multi-Task Learning (NeurIPS 2026 · similarity 0.86)
- Modeling Multi-Task Model Merging as Adaptive Projective Gradient Descent (ICML 2025 · similarity 0.86)
- Bayesian Uncertainty for Gradient Aggregation in Multi-Task Learning (ICML 2024 · similarity 0.86)
- AdaMerging: Adaptive Model Merging for Multi-Task Learning (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
