Research map / Label-efficient and robust learning / Self-supervised and contrastive
Exploring the Gap between Collapsed & Whitened Features in Self-Supervised Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add PostMan-Pat power-law eigenvalue rescaling post-hoc to pretrained encoder | Barlow Twins ResNet-50 encoder · ImageNet-1K 0.3% labels | top-1 accuracy: +2.4 (42.3 vs 39.9) | Helped |
| Add PostMan-Pat post-hoc rescaling to SimCLR encoder | SimCLR ResNet-50 encoder · ImageNet-1K 1% labels | top-1 accuracy: +2.8 over LP (50.9 vs 48.1) | Helped |
| Add PostMan-Pat post-hoc rescaling to SwAV encoder | SwAV ResNet-50 encoder · ImageNet-1K 1% labels | top-1 accuracy: beats NFT by over 2.5% | Helped |
| Add PostMan-Pat post-hoc rescaling to MoCo-v3 ViT-B/16 encoder | ViT-B/16 MoCo-v3 · ImageNet-1K 0.3% labels | top-1 accuracy: +1.2 (55.2 vs 54.0) | Helped |
| Add PostMan-Pat post-hoc rescaling to pretrained encoders for transfer | ResNet-50 SSL encoders (CIFAR-100, Cars, Flowers) · transfer learning, 100% labels | top-1 test accuracy: small gains of about 0.5 to 2.0 points across encoders and datasets | Helped |
| Add PostMan-Pat on supervised encoder (ResNet-18, CIFAR-10) (ablation) | supervised ResNet-18 encoder · CIFAR-10 low labelled data | test accuracy relative to linear probe | No clear effect |
| Increase PMP exponent beta (faster eigenvalue decay) (ablation) | Barlow Twins / SimCLR ResNet-18 encoders · CIFAR-10 with 0.1% and 0.3% labels | test accuracy relative to linear probe: up to 4% higher for Barlow Twins at 0.1% labels | Helped |
| Increase PMP exponent beta (faster eigenvalue decay) (ablation) | Barlow Twins / SimCLR ResNet-18 encoders · CIFAR-10 with 100% labels | test accuracy relative to linear probe | Hurt |
| Use deeper projection MLP (more layers) in Barlow Twins training (ablation) | Barlow Twins ResNet-18 encoder · CIFAR-10 encoder eigenspectrum | encoder normalised eigensum (NESum) | Hurt |
| Increase Barlow Twins regularisation strength ρ (more whitened features) (ablation) | Barlow Twins ResNet-18 encoder · CIFAR-10 linear probe | test accuracy | Mixed |
| Add PMP-style power-law rescaling to deeper-projector encoders (ablation) | encoders trained with deeper projection layers · CIFAR-10 small labelled data | test accuracy gain from PMP | Hurt |
Closest papers by meaning
- Modulate Your Spectrum in Self-Supervised Learning (ICLR 2024 · similarity 0.90)
- Preventing Dimensional Collapse in Self-Supervised Learning via Orthogonality Regularization (NeurIPS 2024 · similarity 0.87)
- Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse (ICLR 2025 · similarity 0.86)
- Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning (NeurIPS 2025 · similarity 0.86)
- Collapse-Proof Non-Contrastive Self-Supervised Learning (ICML 2025 · similarity 0.85)
- Generalized Neural Collapse for a Large Number of Classes (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.