Research map / Label-efficient and robust learning / Self-supervised and contrastive
The Role of Entropy and Reconstruction in Multi-View Self-Supervised Learning
Borja Rodríguez Gálvez, Arno Blaas, Pau Rodriguez, Adam Golinski, Xavier Suau, Jason Ramapuram, Dan Busbridge, Luca Zappella
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace objective with ER bound (SimCLR + ER) | SimCLR · ImageNet linear eval | Top-1 accuracy: +0.72 | Improved |
| add ER bound to BYOL / replace DINO loss with ER | BYOL and DINO · ImageNet linear eval | Top-1 accuracy: -1.5 (BYOL), -1.65 (DINO) | Hurt |
| train with ER bound instead of original loss | BYOL, DINO, SimCLR · ImageNet, batch size 512 degradation | Accuracy drop from bs 4096 to 512: BYOL 23.65 -> 2.35; DINO 6.76 -> 2.35; SimCLR 2.17 -> 1.01 | Improved |
| train with ER bound instead of original loss | BYOL, DINO · ImageNet, EMA coefficient 0.8 | Accuracy drop: DINO 8.25 -> 0.92; BYOL 2.63 -> 0.41 | Improved |
| remove softmax centering from DINO (with and without ER) (ablation) | DINO · ImageNet | entropy and accuracy | Hurt |
| add ER bound to BYOL (entropy maximization check) | BYOL · ImageNet entropy trajectory | entropy during training | No clear effect |
Closest papers by meaning
- Information Flow in Self-Supervised Learning (ICML 2024 · similarity 0.86)
- Matrix Information Theory for Self-Supervised Learning (ICML 2024 · similarity 0.85)
- Clustering Properties of Self-Supervised Learning (ICML 2025 · similarity 0.85)
- Differentiable Information Bottleneck for Deterministic Multi-view Clustering (CVPR 2024 · similarity 0.85)
- Joint‑Embedding vs Reconstruction: Provable Benefits of Latent Space Prediction for Self‑Supervised Learning (NeurIPS 2025 · similarity 0.85)
- Connecting Jensen–Shannon and Kullback–Leibler Divergences: A New Bound for Representation Learning (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
