AI Paper Atlas

Research map / Label-efficient and robust learning / Self-supervised and contrastive

Understanding the Learning Phases in Self-Supervised Learning via Critical Periods

JangHyeon Lee, Philipe Ambrozio Dias, Yao-Yi Chiang, Dalton Lunga

ICLR 2026 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Data-efficient and robust self-supervision

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
select CP closure checkpoint instead of final checkpointVICReg-RN50-fMoW · fMoW-WILDS (OOD)top-1 accuracy: 0.341 -> 0.430Improved
select CP closure checkpoint instead of final checkpointVICReg-RN50-fMoW · fMoW-val (ID)top-1 accuracy: 0.621 -> 0.610Hurt
select CP closure checkpoint instead of final checkpointVICReg-RN50-fMoW · EuroSAT-Spatial (OOD)top-1 accuracy: 0.894 -> 0.912Improved
add CP-guided self-distillation of early layers to final checkpointVICReg-RN50-fMoW · fMoW-WILDS (OOD)top-1 accuracy: 0.341 -> 0.445Improved
add CP-guided self-distillation of early layers to final checkpointVICReg-RN50-fMoW · EuroSAT (OOD)top-1 accuracy: 0.917 -> 0.944Improved
add CP-guided self-distillation of early layers to final checkpointVICReg-RN50-fMoW · fMoW-val (ID)top-1 accuracy: 0.621 -> 0.617No clear effect
distill early layers rather than all layers (ablation)CP-SD · fMoW-WILDS (OOD)top-1 accuracy: 0.421 -> 0.445Improved
distill early layers rather than all layers (ablation)CP-SD · EuroSAT-Spatial (OOD)top-1 accuracy: 0.908 -> 0.925Improved
inject noise deficit early in pretraining rather than lateSSL pretraining (SimCLR, VICReg, DINO, MAE on fMoW) · fMoW-valnormalized sensitivityHurt
increase minibatch reuse from k=1 to k=5 (ablation)DINO pretraining · Fisher InformationFI peak timingMixed
use ViT-Base instead of ViT-Small (ablation)DINO pretraining · Fisher InformationFI stabilizationMixed
pretrain longer (to 1000 epochs) instead of stopping at CP closureSSL pretraining · fMoW-WILDS (OOD)top-1 accuracy: 0.430 -> 0.341Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy