AI Paper Atlas

Research map / Label-efficient and robust learning / Self-supervised and contrastive

A Survey on Self-Supervised Learning: Algorithms, Applications, and Future Trends

Jie Gui, Tuo Chen, Jing Zhang, Qiong Cao, Zhenan Sun, Hao Luo, Dacheng Tao

TPAMI 2024 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Data-efficient and robust self-supervision · 756 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Pretrain MIM with a stronger tokenizer/target (BEiT v2 over BEiT v1)ViT-B MIM · ImageNet-1k linear probetop-1 accuracy: 56.7 -> 80.1Improved
Use MAE pixel reconstruction instead of BEiT tokensViT-B MIM · ImageNet-1k linear probetop-1 accuracy: 56.7 -> 67.8Improved
Add self-distillation plus MIM (iBOT) over MAEViT-B · ImageNet-1k linear probetop-1 accuracy: 67.8 -> 79.5Improved
Replace contrastive pretraining with MIM for fine-tuning (MAE over MoCo v3)ViT-B · ImageNet-1k fine-tuningtop-1 accuracy: 83.2 -> 83.6Improved
Use BEiT v2 over MAE for fine-tuningViT-B · ImageNet-1k fine-tuningtop-1 accuracy: 83.6 -> 85.5Improved
Use MIM (MAE) over contrastive MoCo v3 for COCO detectionViT-B · COCO detectionAP_bb: 47.9 -> 50.3Improved
Use MIM iBOT over MAE on COCO detectionViT-B · COCO detectionAP_bb: 50.3 -> 51.2Improved
Use SSL contrastive MoCo v1 over supervised R50 for COCO detectionResNet-50 · COCO detectionAP_bb: 40.6 -> 40.8Improved
Use ViT-B DINO self-distillation over SimSiam on linear probebackbone differs · ImageNet-1k linear probetop-1 accuracy: 71.3 -> 78.2Improved
Use SSL (DINO) over supervised ViT-B for linear probeViT-B · ImageNet-1k linear probetop-1 accuracy: 82.3 -> 78.2Hurt
Pretrain on K600 instead of K400 for CVRLCVRL R3D50 · UCF101 fine-tuneaccuracy: 92.2 -> 93.4Improved
Pretrain on K600 instead of K400 for BraVeBraVe R3D50 · UCF101 fine-tuneaccuracy: 93.7 -> 94.4Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy