Research map / Label-efficient and robust learning / Self-supervised and contrastive
Learning Decorrelated Representations Efficiently Using Fast Fourier Transform
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace elementwise decorrelation with FFT-based summary regularization | Barlow Twins-style SSL · ImageNet-100 linear evaluation | Top-1 accuracy (%): 80.12 -> 79.94 | No clear effect |
| Add grouped summary regularization with b=128 | Proposed Barlow Twins-style SSL · ImageNet-100 linear evaluation | Top-1 accuracy (%): 79.94 -> 81.02 | Helped |
| Replace elementwise decorrelation with FFT-based summary regularization | Barlow Twins-style SSL · ImageNet linear evaluation, d=8192 | Top-1 accuracy (%): 72.4 -> 73.0 | Helped |
| Replace covariance decorrelation with FFT-based summary regularization | VICReg-style SSL · ImageNet linear evaluation, d=8192 | Top-1 accuracy (%): 72.6 -> 72.8 | No clear effect |
| Replace elementwise decorrelation with FFT-based summary regularization | Barlow Twins-style SSL · ImageNet, 8 GPUs, batch size 1024 | Training time: 6d 14h 0m -> 5d 14h 58m | Helped |
| Add random feature permutation during training (ablation) | Proposed Barlow Twins-style SSL without grouping · ImageNet-100 | Top-1 accuracy (%): 59.64 -> 79.94 | Helped |
| Add random feature permutation during training (ablation) | Proposed VICReg-style SSL without grouping · ImageNet-100 | Top-1 accuracy (%): 57.42 -> 79.20 | Helped |
| Use grouped summary regularization with b=128 instead of no grouping (ablation) | Proposed Barlow Twins-style SSL · ImageNet-100 linear evaluation | Top-1 accuracy (%): 79.94 -> 81.02 | Helped |
| Set regularizer exponent q=2 instead of q=1 (ablation) | Proposed Barlow Twins-style SSL without grouping · ImageNet-100 | Top-1 accuracy (%): 75.94 -> 79.94 | Helped |
| Set regularizer exponent q=2 instead of q=1 (ablation) | Proposed VICReg-style SSL without grouping · ImageNet-100 | Top-1 accuracy (%): 79.20 -> 57.98 | Hurt |
| Use FFT-based summary regularization instead of full covariance decorrelation | Proposed VICReg-style SSL · ImageNet-100, d=8192 | Training speed: 2.8 times as fast as VICReg | Helped |
| Use FFT-based summary regularization instead of full cross-correlation decorrelation | Proposed Barlow Twins-style SSL · ImageNet-100, d=8192 | Training speed: 2.2 times as fast as Barlow Twins | Helped |
Closest papers by meaning
- VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning (ICLR 2022 · similarity 0.86)
- Bridging the Gap from Asymmetry Tricks to Decorrelation Principles in Non-contrastive Self-supervised Learning (NeurIPS 2022 · similarity 0.86)
- LDReg: Local Dimensionality Regularized Self-Supervised Learning (ICLR 2024 · similarity 0.85)
- Preventing Dimensional Collapse in Self-Supervised Learning via Orthogonality Regularization (NeurIPS 2024 · similarity 0.84)
- Fourier Sliced-Wasserstein Embedding for Multisets and Measures (ICLR 2025 · similarity 0.84)
- T-REGS: Minimum Spanning Tree Regularization for Self-Supervised Learning (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.