Research map / Label-efficient and robust learning / Self-supervised and contrastive
Multi-Task Self-Training for Learning General Representations
Golnaz Ghiasi, Barret Zoph, Ekin D. Cubuk, Quoc V. Le, Tsung-Yi Lin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multi-task pseudo labels (depth, segmentation, detection) to ImageNet supervised training | ImageNet-pretrained ResNet-FPN · Pascal detection fine-tuning | AP50: 79.3 -> 85.1 | Improved |
| add multi-task pseudo labels to ImageNet supervised training | ImageNet-pretrained ResNet-FPN · Pascal segmentation fine-tuning | mIoU: 70.6 -> 80.6 | Improved |
| add multi-task pseudo labels to ImageNet supervised training | ImageNet-pretrained ResNet-FPN · NYU depth fine-tuning | % within 1.25: 81.0 -> 87.8 | Improved |
| add depth pseudo labels to ImageNet supervised training (ablation) | ImageNet-pretrained ResNet-FPN · NYU depth vs CIFAR-100 | accuracy / depth metric: NYU depth 81.0 -> 86.0; CIFAR-100 85.4 -> 84.4 | Mixed |
| add segmentation pseudo labels on top of depth pseudo labels (ablation) | MuST on ImageNet · Pascal segmentation fine-tuning | mIoU: 71.0 -> 78.6 | Improved |
| add detection pseudo labels on top of depth and segmentation pseudo labels (ablation) | MuST on ImageNet · Pascal detection fine-tuning | AP50: 81.6 -> 85.1 | Improved |
| train student on Objects365 detection pseudo labels instead of the supervised teacher | ResNet-152 FPN · Objects365 detection AP vs Pascal detection transfer | AP / AP50: Objects365 AP 26.1 -> 20.6; Pascal detection 87.6 -> 86.0 | Mixed |
| use JFT (300M) instead of ImageNet (1.2M) as the unlabeled set for detection pseudo labels | detection pseudo-label student · CIFAR-100 fine-tuning | top-1 accuracy: 83.2 -> 85.2 | Improved |
| add pseudo labels to supervised multi-task training on ImageNet, COCO and Objects365 | supervised multi-task model · ADE segmentation fine-tuning | mIoU: 43.9 -> 45.4 | Improved |
| use MuST pseudo labels on JFT instead of JFT supervised labels (10 epochs) | ResNet-FPN pre-trained on JFT · Pascal detection fine-tuning | AP50: 85.2 -> 87.8 | Improved |
| use MuST pseudo labels on JFT instead of JFT supervised labels (10 epochs) | ResNet-FPN pre-trained on JFT · CIFAR-100 fine-tuning | top-1 accuracy: 89.6 -> 87.6 | Hurt |
| add JFT supervised labels to MuST pseudo labels on JFT | MuST on JFT · CIFAR-100 fine-tuning | top-1 accuracy: 87.6 -> 88.3 | Improved |
Closest papers by meaning
- Multi-Task Self-Supervised Visual Learning (ICCV 2017 · similarity 0.89)
- Learning Representations by Maximizing Mutual Information Across Views (NeurIPS 2019 · similarity 0.88)
- Scaling and Benchmarking Self-Supervised Visual Representation Learning (ICCV 2019 · similarity 0.88)
- Active Multi-Task Representation Learning (ICML 2022 · similarity 0.88)
- Self-supervised Learning from a Multi-view Perspective (ICLR 2021 · similarity 0.87)
- Probing the Mid-level Vision Capabilities of Self-Supervised Learning (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
