Research map / Label-efficient and robust learning / Recognition and long-tail
Joint-Task Regularization for Partially Labeled Multi-Task Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Joint-Task Regularization (JTR), which stacks predictions and labels across all tasks and regularizes them in a single joint-task latent space, for partially labeled multi-task dense prediction with linear complexity in task count.
Problem
Multi-task learning on dense prediction tasks when each image is labeled for only a subset of tasks; prior pairwise cross-task regularization methods scale quadratically with the number of tasks.
Approach
Stack predictions (with predictions substituted for missing labels) across all tasks into one tensor, encode it with an autoencoder into a joint-task latent space, and minimize the latent distance between prediction and target tensors, so regularization is joint rather than pairwise and scales linearly.
Key results
- On NYU-v2 randomlabels, JTR reaches ∆% +21.92 versus MTPSL +19.66 and supervised MTL baseline 0.000.
- On Cityscapes onelabel, JTR reaches ∆% +8.219 versus MTPSL +6.702.
- On Taskonomy halflabels, JTR reaches ∆% +5.075 versus MTPSL +3.160.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use JTR instead of MTPSL | SegNet multi-task model · NYU-v2 onelabel | mIoU: 30.40 -> 31.96 | Helped |
| use JTR instead of MTPSL | SegNet multi-task model · NYU-v2 onelabel | aErr: 0.5926 -> 0.5919 | Helped |
| use JTR instead of MTPSL | SegNet multi-task model · NYU-v2 onelabel | mErr: 31.68 -> 30.80 | Helped |
| use JTR instead of MTPSL | SegNet multi-task model · NYU-v2 randomlabels | mIoU: 35.60 -> 37.08 | Helped |
| use JTR instead of MTPSL | SegNet multi-task model · NYU-v2 randomlabels | aErr: 0.5576 -> 0.5541 | Helped |
| use JTR instead of MTPSL | SegNet multi-task model · NYU-v2 randomlabels | mErr: 29.70 -> 29.44 | Helped |
| use JTR instead of MTPSL | SegNet multi-task model · Cityscapes onelabel | mIoU: 72.09 -> 72.33 | Helped |
| use JTR instead of MTPSL | SegNet multi-task model · Cityscapes onelabel | aErr: 0.0168 -> 0.0163 | Helped |
| use JTR instead of MTPSL | SegNet multi-task model · Taskonomy onelabel | mean relative improvement: +1.973 -> +3.512 | Helped |
| use JTR instead of MTPSL | SegNet multi-task model · Taskonomy randomlabels | mean relative improvement: +1.261 -> +4.095 | Helped |
| use JTR instead of MTPSL | SegNet multi-task model · Taskonomy halflabels | mean relative improvement: +3.160 -> +5.075 | Helped |
| add reconstruction loss (ablation) | JTR · NYU-v2 randomlabels | mIoU: 35.89 -> 37.08 | Helped |
Surprising findings
- JTR without the reconstruction loss still outperforms supervised MTL and MTPSL on NYU-v2 randomlabels (+15.93 ∆%).
- Naive MuST pseudo-labeling performed worse than supervised MTL on the STL teacher stage (-2.378 ∆%), with the MTL student only reaching +8.577.
Limitations
- Only applied to dense prediction tasks; heterogeneous output types (classification, regression) not handled.
- Experiments use single-domain data; combining heterogeneous datasets with domain gaps is untested.
- Does not exploit prior knowledge about known task relations.
Experimental setup
Datasets: NYU-v2 Cityscapes Taskonomy
Benchmarks: NYU-v2 Cityscapes Taskonomy
Metrics: mIoU absolute error mean angle error relative improvement (∆%)
Baselines: Supervised MTL Consistency Regularization MTPSL Direct Cross-Task Mapping Perceptual Cross-Task Mapping MuST
Builds on: MTPSL Consistency Regularization SegNet MuST
Closest papers by meaning
- Learning Multiple Dense Prediction Tasks From Partially Annotated Data (CVPR 2022 · similarity 0.88)
- StableMTL: Repurposing Latent Diffusion Models for Multi-Task Learning from Partially Annotated Synthetic Datasets (CVPR 2026 · similarity 0.86)
- Learning Sparse Task Relations in Multi-Task Learning (AAAI 2017 · similarity 0.86)
- Task Groupings Regularization: Data-Free Meta-Learning with Heterogeneous Pre-trained Models (ICML 2024 · similarity 0.86)
- Multi-task Learning with 3D-Aware Regularization (ICLR 2024 · similarity 0.86)
- Distribution Matching for Multi-Task Learning of Classification Tasks: A Large-Scale Study on Faces & Beyond (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.