Research map / Label-efficient and robust learning / Self-supervised and contrastive
Boosting Self-Supervised Learning via Knowledge Transfer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Transfer VGG16 Jigsaw++ representations to AlexNet using cluster-derived pseudo-labels rather than training Jigsaw++ directly on AlexNet | Jigsaw++ · PASCAL VOC 2007 detection | mAP (SS): 55.5 -> 56.5 | Helped |
| Use VGG16 for Jigsaw++ pre-training, then transfer to AlexNet | Cluster classification with Jigsaw++ · PASCAL VOC 2007 classification | mAP: 69.9 -> 72.5 | Helped |
| Apply cluster classification to Jigsaw++ features trained on AlexNet | Jigsaw++ · PASCAL VOC 2007 classification | mAP: 69.8 -> 69.9 | No clear effect |
| Replace feature regression distillation with cluster pseudo-label transfer | VGG16 Jigsaw++ teacher and AlexNet student · PASCAL VOC 2007 classification | mAP: 58.5 -> 69.9 | Helped |
| Vary the number of clusters from 500 to 10000 (ablation) | CC+Jigsaw++ · PASCAL VOC 2007 classification | mAP: 69.1 -> 69.5 -> 69.9 -> 69.9 -> 70.0 | No clear effect |
| Assign pseudo-labels on Places rather than ImageNet (ablation) | CC+Jigsaw++ with ImageNet clustering · PASCAL VOC 2007 classification | mAP: 69.9 -> 68.4 | Hurt |
| Cluster on Places rather than ImageNet (ablation) | CC+Jigsaw++ with ImageNet pseudo-label assignment · PASCAL VOC 2007 classification | mAP: 69.9 -> 68.3 | Hurt |
| Use Jigsaw++ instead of the original Jigsaw task | AlexNet self-supervised pre-training · PASCAL VOC 2007 detection | mAP (SS): 53.2 -> 55.5 | Helped |
| Use CC+HOG pseudo-labels instead of random initialization | AlexNet · PASCAL VOC 2007 classification | mAP: 53.3 -> 70.2 | Helped |
Closest papers by meaning
- How Well Do Self-Supervised Models Transfer? (CVPR 2021 · similarity 0.89)
- Self-supervised Label Augmentation via Input Transformations (ICML 2020 · similarity 0.88)
- Self-supervised Learning from a Multi-view Perspective (ICLR 2021 · similarity 0.87)
- Revisiting Knowledge Transfer for Training Object Class Detectors (CVPR 2018 · similarity 0.87)
- Knowledge Transfer from Vision Foundation Models for Efficient Training of Small Task-specific Models (ICML 2024 · similarity 0.86)
- How Well Does Self-Supervised Pre-Training Perform with Streaming Data? (ICLR 2022 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.