Research map / Label-efficient and robust learning / Self-supervised and contrastive
An Investigation into Whitening Loss for Self-supervised Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add random group partition (CW to CW-RGP), 4 views (ablation) | CW 4 · CIFAR-100 | linear accuracy: 66.90 -> 68.26 | Helped |
| add random group partition (CW to CW-RGP), 4 views (ablation) | CW 4 · CIFAR-10 | linear accuracy: 92.10 -> 92.47 | Helped |
| add random group partition, 2 views (ablation) | CW 2 · CIFAR-100 | linear accuracy: 66.26 -> 67.51 | Helped |
| use non-random group partition instead of none (4 views) (ablation) | CW 4 · CIFAR-10 | linear accuracy: 92.10 -> 92.08 | No clear effect |
| use non-random group partition instead of none (2 views) (ablation) | CW 2 · CIFAR-10 | linear accuracy: 91.66 -> 91.61 | No clear effect |
| use random instead of fixed group partition (4 views) (ablation) | CW-GP 4 · CIFAR-100 | 5-nn accuracy: 57.28 -> 58.67 | Helped |
| use 4 views instead of 2 views (ablation) | CW-RGP · CIFAR-100 | linear accuracy: 67.51 -> 68.26 | Helped |
| use 4 views instead of 2 views (ablation) | CW-RGP · Tiny-ImageNet | linear accuracy: 49.23 -> 50.24 | Helped |
| replace batch whitening (W-MSE 4) with CW-RGP 4 (ablation) | CW-RGP 4 · CIFAR-100 | linear accuracy: 67.64 -> 68.26 | Helped |
| replace batch whitening with CW-RGP 4 on Tiny-ImageNet | CW-RGP 4 · Tiny-ImageNet | linear accuracy: 51.00 -> 50.24 | Hurt |
| CW-RGP 4 vs reproduced W-MSE 4 at batch size 512 | CW-RGP 4 · ImageNet 100 epochs | top-1 accuracy: 66.7 -> 69.7 | Helped |
| batch size 256 instead of 512 for CW-RGP 100 epochs (ablation) | CW-RGP 4 · ImageNet 100 epochs | top-1 accuracy: 69.7 -> 69.5 | No clear effect |
Closest papers by meaning
- Modulate Your Spectrum in Self-Supervised Learning (ICLR 2024 · similarity 0.91)
- Whitening for Self-Supervised Representation Learning (ICML 2021 · similarity 0.90)
- Collapse-Proof Non-Contrastive Self-Supervised Learning (ICML 2025 · similarity 0.86)
- Preventing Dimensional Collapse in Self-Supervised Learning via Orthogonality Regularization (NeurIPS 2024 · similarity 0.85)
- Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning (NeurIPS 2025 · similarity 0.83)
- Harnessing small projectors and multiple views for efficient vision pretraining (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.