Research map / Label-efficient and robust learning / Self-supervised and contrastive
Directional Neural Collapse Explains Few-Shot Transfer in Self-Supervised Learning
Achleshwar Luthra, Yash Salunkhe, Tomer Galanti
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| track directional CDNV instead of classical CDNV during SSL pretraining | MAE, SimCLR, DINO-v2, VICReg · mini-ImageNet (train and test) | directional CDNV: from roughly 2^-1 to 2^1 down to about 2^-3 to 2^-5 | Improved |
| measure classical CDNV during SSL pretraining | MAE, SimCLR, DINO-v2, VICReg · mini-ImageNet | classical CDNV | No clear effect |
| use the proposed finite-shot directional bound instead of the prior directional bound | NCC on frozen SSL encoders · mini-ImageNet binary tasks | few-shot error certificate | Improved |
| use the proposed bound instead of Galanti et al. 2023 bound | NCC and LP few-shot error · mini-ImageNet | bound tightness | Improved |
| train SSL longer on decision-axis variance | SSL encoders · mini-ImageNet class pairs | decision-axis variance | Improved |
| train SSL on synthetic multi-factor images | SimCLR ResNet-18, VICReg ResNet-50 · synthetic color/background/shape/size labelings | cosine similarity between decision axes | Improved |
| train VICReg on real attribute data | VICReg ResNet-50 · CelebA, CUB-200 | cosine similarity between decision axes | Improved |
Closest papers by meaning
- The Impact of Geometric Complexity on Neural Collapse in Transfer Learning (NeurIPS 2024 · similarity 0.88)
- Neural Collapse Beyond the Unconstrained Features Model: Landscape, Dynamics, and Generalization in the Mean-Field Regime (ICML 2025 · similarity 0.86)
- Provable Multi-Task Representation Learning by Two-Layer ReLU Neural Networks (ICML 2024 · similarity 0.85)
- No Wrong Turns: The Simple Geometry Of Neural Networks Optimization Paths (ICML 2024 · similarity 0.85)
- Disentangling Representations through Multi-task Learning (ICLR 2025 · similarity 0.85)
- Task structure and nonlinearity jointly determine learned representational geometry (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
