Research map / Label-efficient and robust learning / Self-supervised and contrastive
Whitening for Self-Supervised Representation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use W-MSE with four augmented views per image instead of two (ablation) | W-MSE · CIFAR-100 | linear accuracy: 66.10 -> 67.64 | Helped |
| Use W-MSE with four augmented views per image instead of two (ablation) | W-MSE · ImageNet-100 | top-1 accuracy: 76.00 -> 79.02 | Helped |
| Use W-MSE with four augmented views per image instead of two (ablation) | W-MSE · CIFAR-10 | linear accuracy: 91.55 -> 91.99 | Helped |
| Use W-MSE with four augmented views per image instead of two (ablation) | W-MSE · STL-10 | linear accuracy: 90.36 -> 91.75 | Mixed |
| Add whitening to the contrastive loss while retaining L2 normalization (ablation) | Contrastive loss · CIFAR-10 | training convergence: diverged | Hurt |
| Add whitening to the contrastive loss and omit L2 normalization (ablation) | Contrastive loss · CIFAR-10 | linear accuracy: 89.66 -> 77.39 | Hurt |
| Replace cosine distance with Euclidean distance and use unnormalized embeddings (ablation) | W-MSE 4 · STL-10 | linear accuracy: 91.75 -> 90.40 | No clear effect |
| Use W-MSE instead of BYOL | ResNet-18 SSL · Tiny ImageNet | linear accuracy: 51.00 -> 49.22 | Hurt |
| Use W-MSE instead of SimCLR | ResNet-50 SSL · ImageNet, 100 epochs | top-1 accuracy: 66.5 -> 69.43 | Helped |
| Use W-MSE instead of BYOL | ResNet-50 SSL · ImageNet, 400 epochs | top-1 accuracy: 73.2 -> 72.56 | Hurt |
| Use batch standardization instead of full whitening in the positive-only MSE setup (ablation) | W-MSE · CIFAR-10 | linear classification accuracy: 68.15% | Hurt |
| Use W-MSE instead of BYOL during early training | SSL representation learning · STL-10, 20 epochs | linear accuracy: 79.98% vs. 73.24% | Helped |
Closest papers by meaning
- An Investigation into Whitening Loss for Self-supervised Learning (NeurIPS 2022 · similarity 0.90)
- Modulate Your Spectrum in Self-Supervised Learning (ICLR 2024 · similarity 0.89)
- Decoupled Contrastive Learning for Long-Tailed Recognition (AAAI 2024 · similarity 0.85)
- Collapse-Proof Non-Contrastive Self-Supervised Learning (ICML 2025 · similarity 0.85)
- Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning (NeurIPS 2025 · similarity 0.84)
- Views Can Be Deceiving: Improved SSL Through Feature Space Augmentation (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.