Research map / Label-efficient and robust learning / Self-supervised and contrastive
Reverse Engineering Self-Supervised Learning
Ido Ben-Shaul, Ravid Shwartz-Ziv, Tomer Galanti, Shai Dekel, Yann LeCun
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| larger regularization coefficient mu (25 instead of 5) in VICReg (ablation) | VICReg RES-5-250 · CIFAR-100 linear probing | linear test accuracy | Improved |
| increase regularization coefficient mu from 25 to 100 (ablation) | VICReg RES-5-250 · CIFAR-100 linear probing | linear test accuracy | No clear effect |
| SSL (VICReg) versus supervised training | RES-10-250 · CIFAR-100 NCC test accuracy | NCC test accuracy | No clear effect |
| add data augmentation to supervised training | supervised RES-10-250 · CIFAR-100 NCC accuracy | NCC test accuracy | Improved |
| VICReg instead of SimCLR | RES-5-250 · CIFAR-100 intermediate-layer linear accuracy | linear test accuracy | Improved |
| wider network (width 50, 250, 1000) (ablation) | RES-5 VICReg · CIFAR-100 linear probing | linear test accuracy | Improved |
| deeper network (RES-10-250 vs RES-5-250) (ablation) | VICReg · CIFAR-100 linear probing | linear test accuracy | Mixed |
Closest papers by meaning
- Clustering Properties of Self-Supervised Learning (ICML 2025 · similarity 0.91)
- The SSL Interplay: Augmentations, Inductive Bias, and Generalization (ICML 2023 · similarity 0.88)
- Memorization in Self-Supervised Learning Improves Downstream Generalization (ICLR 2024 · similarity 0.86)
- On the Importance of Embedding Norms in Self-Supervised Learning (ICML 2025 · similarity 0.85)
- Exploring Structural Degradation in Dense Representations for Self-supervised Learning (NeurIPS 2025 · similarity 0.85)
- Self Iterative Label Refinement via Robust Unlabeled Learning (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
