Research map / Label-efficient and robust learning / Self-supervised and contrastive
VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add variance regularization to BYOL (ablation) | BYOL · ImageNet linear, 100 pretraining epochs | Top-1 accuracy: +0.9% | Helped |
| Add variance regularization to SimSiam (ablation) | SimSiam · ImageNet linear, 100 pretraining epochs | Top-1 accuracy | No clear effect |
| Add a predictor to VICReg when variance regularization is already used (ablation) | VICReg · ImageNet linear, 100 pretraining epochs | Top-1 accuracy | No clear effect |
| Add batch normalization in the expander hidden layers (ablation) | VICReg · ImageNet linear, 100 pretraining epochs | Top-1 accuracy: +1.0% | Helped |
| Add variance regularization to SimSiam without batch normalization (ablation) | SimSiam · ImageNet linear, 100 pretraining epochs | Top-1 accuracy: 35.1 -> 67.3 | Helped |
| Add covariance regularization alongside variance regularization to VICReg (ablation) | VICReg · ImageNet linear, 100 pretraining epochs | Top-1 accuracy: 57.5 -> 68.6 | Helped |
| Remove variance regularization (ablation) | VICReg · ImageNet linear, 100 pretraining epochs | Top-1 accuracy: 68.6 -> collapse | Hurt |
| Use loss coefficients λ=1, μ=1, ν=1 (ablation) | VICReg · ImageNet linear, 100 pretraining epochs | Top-1 accuracy: 68.6 -> collapse | Hurt |
| Increase expander dimensionality from 256 to 16384 (ablation) | VICReg · ImageNet linear, 100 pretraining epochs | Top-1 accuracy: 55.9 -> 68.8 | Helped |
| Reduce batch size to 128 from the 2048 setting (ablation) | VICReg · ImageNet linear, 100 pretraining epochs | Top-1 accuracy: 68.6 -> 67.3 | Hurt |
| Standardize the embeddings (ablation) | VICReg · ImageNet linear, 100 pretraining epochs | Top-1 accuracy: 68.6 -> 68.4 | Hurt |
| L2-normalize the embeddings and set the variance target to 1/√d (ablation) | VICReg · ImageNet linear, 100 pretraining epochs | Top-1 accuracy: 68.6 -> 65.1; drops by 3.5% | Hurt |
Closest papers by meaning
- An Information Theory Perspective on Variance-Invariance-Covariance Regularization (NeurIPS 2023 · similarity 0.90)
- VICRegL: Self-Supervised Learning of Local Visual Features (NeurIPS 2022 · similarity 0.89)
- Self-supervised learning with rotation-invariant kernels (ICLR 2023 · similarity 0.88)
- LDReg: Local Dimensionality Regularized Self-Supervised Learning (ICLR 2024 · similarity 0.87)
- LORE: Lagrangian-Optimized Robust Embeddings for Visual Encoders (NeurIPS 2025 · similarity 0.85)
- Channel-wise Vector Quantization (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.