AI Paper Atlas

Research map / Label-efficient and robust learning / Self-supervised and contrastive

Multi-Task Self-Training for Learning General Representations

Golnaz Ghiasi, Barret Zoph, Ekin D. Cubuk, Quoc V. Le, Tsung-Yi Lin

ICCV 2021 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Invariance-based self-supervised learning · 89 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add multi-task pseudo labels (depth, segmentation, detection) to ImageNet supervised trainingImageNet-pretrained ResNet-FPN · Pascal detection fine-tuningAP50: 79.3 -> 85.1Improved
add multi-task pseudo labels to ImageNet supervised trainingImageNet-pretrained ResNet-FPN · Pascal segmentation fine-tuningmIoU: 70.6 -> 80.6Improved
add multi-task pseudo labels to ImageNet supervised trainingImageNet-pretrained ResNet-FPN · NYU depth fine-tuning% within 1.25: 81.0 -> 87.8Improved
add depth pseudo labels to ImageNet supervised training (ablation)ImageNet-pretrained ResNet-FPN · NYU depth vs CIFAR-100accuracy / depth metric: NYU depth 81.0 -> 86.0; CIFAR-100 85.4 -> 84.4Mixed
add segmentation pseudo labels on top of depth pseudo labels (ablation)MuST on ImageNet · Pascal segmentation fine-tuningmIoU: 71.0 -> 78.6Improved
add detection pseudo labels on top of depth and segmentation pseudo labels (ablation)MuST on ImageNet · Pascal detection fine-tuningAP50: 81.6 -> 85.1Improved
train student on Objects365 detection pseudo labels instead of the supervised teacherResNet-152 FPN · Objects365 detection AP vs Pascal detection transferAP / AP50: Objects365 AP 26.1 -> 20.6; Pascal detection 87.6 -> 86.0Mixed
use JFT (300M) instead of ImageNet (1.2M) as the unlabeled set for detection pseudo labelsdetection pseudo-label student · CIFAR-100 fine-tuningtop-1 accuracy: 83.2 -> 85.2Improved
add pseudo labels to supervised multi-task training on ImageNet, COCO and Objects365supervised multi-task model · ADE segmentation fine-tuningmIoU: 43.9 -> 45.4Improved
use MuST pseudo labels on JFT instead of JFT supervised labels (10 epochs)ResNet-FPN pre-trained on JFT · Pascal detection fine-tuningAP50: 85.2 -> 87.8Improved
use MuST pseudo labels on JFT instead of JFT supervised labels (10 epochs)ResNet-FPN pre-trained on JFT · CIFAR-100 fine-tuningtop-1 accuracy: 89.6 -> 87.6Hurt
add JFT supervised labels to MuST pseudo labels on JFTMuST on JFT · CIFAR-100 fine-tuningtop-1 accuracy: 87.6 -> 88.3Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy