Research map / Label-efficient and robust learning / Self-supervised and contrastive
Mean Shift for Self-Supervised Learning
Soroush Abbasi Koohpayegani, Ajinkya Tejankar, Hamed Pirsiavash
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use weak augmentation for the target view (w/s) instead of strong on both views (ablation) | MSF · ImageNet linear eval | top-1 accuracy: 71.4 -> 72.4 | Improved |
| use weak augmentation for the target view (w/s) (ablation) | BYOL-asym · ImageNet linear eval | top-1 accuracy: 69.3 -> 69.5 | No clear effect |
| use nearest neighbors in the target (MSF vs BYOL) with weak/weak augmentation | MSF w/w · ImageNet linear eval | top-1 accuracy: 60.1 -> 66.3 | Improved |
| use 5 nearest neighbors as targets instead of only 1 (BYOL-asym) (ablation) | MSF · ImageNet 1-NN | top-1 accuracy: 55.8 -> 62.0 | Improved |
| use 5 nearest neighbors as targets instead of only 1 (BYOL-asym) (ablation) | MSF · ImageNet 20-NN | top-1 accuracy: 59.1 -> 64.9 | Improved |
| increase k from 5 to 10 (ablation) | MSF w/s · ImageNet 20-NN | top-1 accuracy: 64.9 -> 65.7 | Improved |
| increase k from 5 to 50 (ablation) | MSF w/s · ImageNet 1-NN | top-1 accuracy: 62.0 -> 61.5 | Hurt |
| reduce memory bank from 1M to 128K (ablation) | MSF w/s · ImageNet 20-NN | top-1 accuracy: 64.9 -> 65.2 | No clear effect |
| reduce memory bank from 1M to 128K (ablation) | MSF w/s · Linear transfer, 10 datasets | mean accuracy: 75.5 -> 76.3 | Improved |
| feed the same augmented view to both encoders (T1=T2) (ablation) | MSF · ImageNet | training stability | Hurt |
| replace BYOL-asym by MSF (adding NN grouping), strong augmentation | BYOL-asym · ImageNet linear eval | top-1 accuracy: 69.3 -> 71.4 | Improved |
| replace BYOL-asym by MSF (adding NN grouping) | BYOL-asym · ImageNet 1% labels | top-1 accuracy: 47.9 -> 53.5 | Improved |
Closest papers by meaning
- Constrained Mean Shift Using Distant Yet Related Neighbors for Representation Learning (ECCV 2022 · similarity 0.90)
- RSA: Reducing Semantic Shift from Aggressive Augmentations for Self-supervised Learning (NeurIPS 2022 · similarity 0.89)
- Clustering Properties of Self-Supervised Learning (ICML 2025 · similarity 0.86)
- Can Generative Models Improve Self-Supervised Representation Learning? (AAAI 2025 · similarity 0.86)
- Views Can Be Deceiving: Improved SSL Through Feature Space Augmentation (ICLR 2024 · similarity 0.86)
- Harnessing small projectors and multiple views for efficient vision pretraining (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
