Research map / Label-efficient and robust learning / Self-supervised and contrastive
A Survey on Self-Supervised Learning: Algorithms, Applications, and Future Trends
Jie Gui, Tuo Chen, Jing Zhang, Qiong Cao, Zhenan Sun, Hao Luo, Dacheng Tao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Pretrain MIM with a stronger tokenizer/target (BEiT v2 over BEiT v1) | ViT-B MIM · ImageNet-1k linear probe | top-1 accuracy: 56.7 -> 80.1 | Improved |
| Use MAE pixel reconstruction instead of BEiT tokens | ViT-B MIM · ImageNet-1k linear probe | top-1 accuracy: 56.7 -> 67.8 | Improved |
| Add self-distillation plus MIM (iBOT) over MAE | ViT-B · ImageNet-1k linear probe | top-1 accuracy: 67.8 -> 79.5 | Improved |
| Replace contrastive pretraining with MIM for fine-tuning (MAE over MoCo v3) | ViT-B · ImageNet-1k fine-tuning | top-1 accuracy: 83.2 -> 83.6 | Improved |
| Use BEiT v2 over MAE for fine-tuning | ViT-B · ImageNet-1k fine-tuning | top-1 accuracy: 83.6 -> 85.5 | Improved |
| Use MIM (MAE) over contrastive MoCo v3 for COCO detection | ViT-B · COCO detection | AP_bb: 47.9 -> 50.3 | Improved |
| Use MIM iBOT over MAE on COCO detection | ViT-B · COCO detection | AP_bb: 50.3 -> 51.2 | Improved |
| Use SSL contrastive MoCo v1 over supervised R50 for COCO detection | ResNet-50 · COCO detection | AP_bb: 40.6 -> 40.8 | Improved |
| Use ViT-B DINO self-distillation over SimSiam on linear probe | backbone differs · ImageNet-1k linear probe | top-1 accuracy: 71.3 -> 78.2 | Improved |
| Use SSL (DINO) over supervised ViT-B for linear probe | ViT-B · ImageNet-1k linear probe | top-1 accuracy: 82.3 -> 78.2 | Hurt |
| Pretrain on K600 instead of K400 for CVRL | CVRL R3D50 · UCF101 fine-tune | accuracy: 92.2 -> 93.4 | Improved |
| Pretrain on K600 instead of K400 for BraVe | BraVe R3D50 · UCF101 fine-tune | accuracy: 93.7 -> 94.4 | Improved |
Closest papers by meaning
- A Closer Look at Benchmarking Self-supervised Pre-training with Image Classification (IJCV 2025 · similarity 0.91)
- Self-Supervised Learning for Text Recognition: A Critical Survey (IJCV 2025 · similarity 0.90)
- Self-Supervised Learning of Graph Neural Networks: A Unified Review (TPAMI 2022 · similarity 0.90)
- Self-Supervised Classification Network (ECCV 2022 · similarity 0.90)
- Clustering Properties of Self-Supervised Learning (ICML 2025 · similarity 0.88)
- Learning from Memory: Non-Parametric Memory Augmented Self-Supervised Learning of Visual Features (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
