Research map / Label-efficient and robust learning / Self-supervised and contrastive
Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use EMA target network (tau=0.99) instead of constant random target (ablation) | BYOL · ImageNet linear eval | top-1 accuracy: 18.8 -> 72.5 | Helped |
| use EMA target instead of instantaneous copy (tau=0) (ablation) | BYOL · ImageNet linear eval | top-1 accuracy: 0.3 -> 72.5 | Helped |
| tau_base 0.99 vs 0.999 or 0.9 (ablation) | BYOL · ImageNet linear eval | top-1 accuracy: 69.8 / 68.4 -> 72.5 | Helped |
| add predictor to online network (ablation) | BYOL (no negatives) · ImageNet linear eval | top-1 accuracy: 0.2 -> 72.5 | Helped |
| add negative pairs (beta=1) without retuning temperature (ablation) | BYOL · ImageNet linear eval | top-1 accuracy: 72.5 -> 70.9 | Hurt |
| add target network (ablation) | SimCLR · ImageNet linear eval | top-1 accuracy: 69.4 -> 70.7 | Helped |
| add predictor (ablation) | SimCLR · ImageNet linear eval | top-1 accuracy: 69.4 -> 69.1 | No clear effect |
| remove color distortions from augmentations (ablation) | BYOL vs SimCLR · ImageNet linear eval | top-1 accuracy: -9.1 BYOL, -22.2 SimCLR | Hurt |
| reduce augmentations to random crops only (ablation) | BYOL · ImageNet linear eval | top-1 accuracy: 72.5 -> 59.4 | Hurt |
| reduce batch size from 4096 down to 256 (ablation) | BYOL · ImageNet linear eval | top-1 accuracy | No clear effect |
| replace negative-pair contrastive learning with BYOL bootstrapping | ResNet-50 · ImageNet linear eval | top-1 accuracy: 73.0 -> 74.3 | Helped |
| logit regularization in linear eval (ablation) | BYOL ResNet-50 · ImageNet linear eval | top-1 accuracy: 74.3 -> 74.7 | Helped |
Closest papers by meaning
- Understanding Self-Supervised Learning via Latent Distribution Matching (ICML 2026 · similarity 0.85)
- The Edge of Orthogonality: A Simple View of What Makes BYOL Tick (ICML 2023 · similarity 0.85)
- Bridging the Gap from Asymmetry Tricks to Decorrelation Principles in Non-contrastive Self-supervised Learning (NeurIPS 2022 · similarity 0.85)
- Large-Scale Representation Learning on Graphs via Bootstrapping (ICLR 2022 · similarity 0.85)
- L2B: Learning to Bootstrap Robust Models for Combating Label Noise (CVPR 2024 · similarity 0.84)
- Latent Zoning Network: A Unified Principle for Generative Modeling, Representation Learning, and Classification (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.