Research map / Label-efficient and robust learning / Self-supervised and contrastive
Investigating Self-Supervised Methods for Label-Efficient Learning
Srinivasa Rao Nandam, Sara Atito, Zhenhua Feng, Josef Kittler, Muhammed Awais
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add MIM pixel reconstruction to clustering (ablation) | ViT-S clustering (ME-MAX) · ImageNet-1K 1-shot | top-1 accuracy: 6.2 -> 15.7 | Improved |
| add MIM pixel reconstruction to contrastive learning (ablation) | ViT-S contrastive · ImageNet-1K 1-shot | top-1 accuracy: 2.3 -> 6.4 | Improved |
| use clustering instead of contrastive learning (ablation) | ViT-S · ImageNet-1K 1-shot | top-1 accuracy: 2.3 -> 6.2 | Improved |
| use MIM alone instead of clustering alone (ablation) | ViT-S · ImageNet-1K 1-shot | top-1 accuracy: 6.2 -> 8.7 | Improved |
| use ME-MAX instead of centring for collapse avoidance (ablation) | ViT-S clustering · ImageNet-1K 1-shot | top-1 accuracy: 5.3 -> 6.2 | Improved |
| use ME-MAX instead of sinkhorn for collapse avoidance (ablation) | ViT-S clustering · ImageNet-1K 5-shot | top-1 accuracy: 6.5 -> 29.6 | Improved |
| add patch-level clustering to class-level clustering (ablation) | ViT-S clustering · ImageNet-1K 1-shot | top-1 accuracy: 6.2 -> 7.9 | Improved |
| add patch-level clustering to class-level clustering (ablation) | ViT-S clustering · ImageNet-1K 5-shot | top-1 accuracy: 29.6 -> 35.0 | Improved |
| use MaskCluster instead of best prior SSL method | ViT-S · ImageNet-1K 1 image per class | top-1 accuracy: 7.9 -> 22.4 | Improved |
| use MaskCluster instead of MSN for FPTrans initialisation | FPTrans ViT-S/16 · Pascal 5i 1-shot | mIoU: 49.7 -> 63.5 | Improved |
| use MaskCluster instead of iBoT for full ImageNet-1K fine-tuning | ViT-S/16 · ImageNet-1K full | top-1 accuracy: 82.3 -> 82.1 | No clear effect |
| use MaskCluster instead of iBoT for Cars fine-tuning | ViT-S/16 · Cars | top-1 accuracy: 94.0 -> 93.6 | Hurt |
Closest papers by meaning
- Masked Unsupervised Self-training for Label-free Image Classification (ICLR 2023 · similarity 0.89)
- Boosting Few-Shot Visual Learning With Self-Supervision (ICCV 2019 · similarity 0.89)
- Distilling Self-Supervised Vision Transformers for Weakly-Supervised Few-Shot Classification & Segmentation (CVPR 2023 · similarity 0.88)
- Supervised Masked Knowledge Distillation for Few-Shot Transformers (CVPR 2023 · similarity 0.88)
- Towards Understanding Self-Pretraining for Sequence Classification (NeurIPS 2026 · similarity 0.87)
- Probing the Mid-level Vision Capabilities of Self-Supervised Learning (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
