Research map / Detection and segmentation / Open-set and few-shot detection
Rethinking Pre-training and Self-training
Barret Zoph, Golnaz Ghiasi, Tsung-Yi Lin, Yin Cui, Hanxiao Liu, Ekin Dogus Cubuk, Quoc V. Le
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| ImageNet supervised pre-training under strong augmentation | RetinaNet EfficientNet-B7 · COCO 100% Augment-S4 | AP: 44.3 -> 43.3 | Hurt |
| ImageNet supervised pre-training under weak augmentation | RetinaNet EfficientNet-B7 · COCO 100% Augment-S1 | AP: 39.2 -> 39.5 | Improved |
| add self-training on ImageNet pseudo labels | RetinaNet EfficientNet-B7 Rand Init · COCO all augmentation strengths | AP: +1.3 to +1.7 AP | Improved |
| add self-training on top of ImageNet pre-training | ImageNet Init · COCO 20% Augment-S4 | AP: 33.3 -> 36.0 | Improved |
| ImageNet pre-training in low-data regime | RetinaNet EfficientNet-B7 · COCO 20% Augment-S4 | AP: 30.7 -> 33.3 | Improved |
| SimCLR self-supervised pre-training | ResNet-50 RetinaNet · COCO 100% Augment-S4 | AP: 41.1 -> 40.4 | Hurt |
| add self-training with Open Images | SpineNet-190 · COCO test-dev | AP: 52.8 -> 54.3 | Improved |
| add self-training with PASCAL aug | Eff-L2 NAS-FPN · PASCAL VOC 2012 val | mIOU: 88.7 -> 90.0 | Improved |
| add noisy human-labeled aug set under strong augmentation | EfficientNet-B7 ImageNet Init · PASCAL VOC 2012 val Augment-S4 | mIOU: 85.2 -> 84.8 | Hurt |
| joint training with ImageNet classification | Rand Init · COCO 20% Augment-S4 | AP: 30.7 -> 33.6 | Improved |
| use OID instead of ImageNet as unlabeled source (ablation) | self-training EfficientNet-B7 · COCO Augment-S4 | AP: 45.6 -> 46.0 | Improved |
| initialize student from teacher instead of random (ablation) | self-training EfficientNet-B7 · COCO Augment-S4 | AP: 45.2 -> 45.6 | Improved |
Closest papers by meaning
- Rethinking ImageNet Pre-Training (ICCV 2019 · similarity 0.94)
- Towards All-in-One Pre-Training via Maximizing Multi-Modal Mutual Information (CVPR 2023 · similarity 0.89)
- Efficient Visual Pretraining With Contrastive Detection (ICCV 2021 · similarity 0.88)
- The Effectiveness of MAE Pre-Pretraining for Billion-Scale Pretraining (ICCV 2023 · similarity 0.88)
- How Useful Is Self-Supervised Pretraining for Visual Tasks? (CVPR 2020 · similarity 0.88)
- Learning Rich Features at High-Speed for Single-Shot Object Detection (ICCV 2019 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
