Research map / Label-efficient and robust learning / Recognition and long-tail
Identifying and Mitigating Spurious Correlation in Multi-Task Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes identifying spurious inter-task label correlations via correlation-coefficient changes under class-wise resampling, then fine-tuning per-task projectors and predictors with debiased adversarial training against identified spurious tasks.
Problem
Multi-task learning can hurt generalization because spurious correlations between task labels cause per-task predictors to rely on spurious features; existing distributionally robust or joint-distribution approaches lack guarantees and are sensitive to validation-set construction or infeasible for h
Approach
Identify spurious task pairs by measuring the difference in Pearson correlation between the empirical training distribution and class-wise resampled distributions (with a theorem giving a lower bound), then adversarially fine-tune each task's projector and predictor to maximize loss on identified spurious tasks while minimizing its own loss.
Key results
- On Multi-MNIST, the proposed method improves left and right digit accuracy by +9.3% and +9.6% over vanilla MTL, versus +5.9% and +7.2% for GMTL.
- On CelebA, the method improves Gender accuracy by +5.7% over vanilla MTL, the best among compared methods.
- Class-wise resampling correlation change is 0.29 under spurious correlation versus 0.002 under random assignment on Multi-MNIST, supporting the identification criterion.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use debiased adversarial training with identified spurious tasks | MTL · Multi-MNIST | accuracy relative improvement over vanilla MTL: +9.3% (left digit); +9.6% (right digit) | Helped |
| Use debiased adversarial training with identified spurious tasks | MTL · CelebA | accuracy relative improvement over vanilla MTL: +5.7% (Gender); +4.6% (Smiling); +3.9% (Age); +2.4% (Attractiveness) | Helped |
| Use debiased adversarial training with identified spurious tasks | MTL · Taskonomy | relative improvement over vanilla MTL: +9.9% accuracy (Scene); +12.7% accuracy (Object); +6.2% cross entropy (Segmentation) | Mixed |
| Use debiased adversarial training with identified spurious tasks | MTL · MetaShift | accuracy relative improvement over vanilla MTL: +5.4% (Animal); +4.0% (Scene) | Helped |
| Use debiased adversarial training with identified spurious tasks | MTL · NYU-V2 | relative improvement over vanilla MTL: +11.3% mean IoU (Segmentation); +10.2% relative error (Depth) | Mixed |
| Apply adversarial training without identifying spurious tasks, treating the other task as spurious (ablation) | Debiased adversarial training · Multi-MNIST Id.A. | accuracy: 93.67±1.36% -> 91.73±1.45% (left digit); 93.68±1.37% -> 91.85±1.46% (right digit) | Hurt |
| Identify spurious tasks before adversarial training rather than treating the other task as spurious (ablation) | Debiased adversarial training · Multi-MNIST Id.A. | accuracy: 91.73±1.45% -> 93.84±1.43% (left digit); 91.85±1.46% -> 93.52±1.19% (right digit) | Helped |
| Identify spurious tasks before adversarial training (ablation) | Debiased adversarial training · Multi-MNIST Sp.C. | accuracy: 91.59±1.47% -> 91.63±1.46% (left digit); 90.74±1.22% -> 90.78±1.26% (right digit) | No clear effect |
| Identify spurious tasks before adversarial training (ablation) | Debiased adversarial training · Multi-MNIST R.A. | accuracy: 93.48±1.76% -> 93.87±1.24% (left digit); 93.25±1.29% -> 93.73±1.36% (right digit) | No clear effect |
Surprising findings
- Distributional-robustness baselines such as MT-CRL with a balanced or less-biased validation set still underperform vanilla MTL with the optimal setting, and class-balanced meta-learning shows little improvement over ERM.
- Applying debiased adversarial training without correct identification of spurious tasks degrades performance below the STL and MTL baselines under identical label assignment.
Limitations
- Identification relies on binary or multi-class label correlations and a threshold delta that must be chosen; the theory is stated for binary tasks with extension via algorithm.
- GMTL and some methods are not applicable to high-dimensional mask labels, so the comparison is incomplete on dense prediction datasets.
- Experiments on synthetic and Multi-MNIST use constructed spurious correlations, so real-world generality is only partially shown.
Experimental setup
Datasets: Multi-MNIST CelebA Taskonomy MetaShift NYU-V2 CityScape
Benchmarks: Multi-MNIST CelebA Taskonomy MetaShift NYU-V2
Metrics: accuracy mean IoU relative error cross entropy relative improvement over MTL
Baselines: STL MTL SubSel MT-CRL Meta-learning GMTL
Builds on: MT-CRL Meta-learning for MTL generalization GMTL SubSel
Closest papers by meaning
- MTL-UE: Learning to Learn Nothing for Multi-Task Learning (ICML 2025 · similarity 0.87)
- Let Samples Speak: Mitigating Spurious Correlation by Exploiting the Clusterness of Samples (CVPR 2025 · similarity 0.86)
- Distribution Matching for Multi-Task Learning of Classification Tasks: A Large-Scale Study on Faces & Beyond (AAAI 2024 · similarity 0.86)
- NTKMTL: Mitigating Task Imbalance in Multi-Task Learning from Neural Tangent Kernel Perspective (NeurIPS 2025 · similarity 0.85)
- Inductive biases of multi-task learning and finetuning: multiple regimes of feature reuse (NeurIPS 2024 · similarity 0.85)
- Complexity Matters: Feature Learning in the Presence of Spurious Correlations (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.