Research map / Label-efficient and robust learning / Self-supervised and contrastive
IdEst: Assessing Self-Supervised Learning Representations via Intrinsic Dimension
Julie Mordacq, Vicky Kalogeiton, Steve Oudot
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add IdEst-based selection instead of picking at random from the hyperparameter pool | I-JEPA ViT-B · ImageNet, all hyperparameters | top-1 accuracy of selected model: 49.0 -> 66.4 | Improved |
| select with IdEst rather than RankMe | I-JEPA ViT-B · ImageNet, all hyperparameters | top-1 accuracy of selected model: 61.9 -> 66.4 | Improved |
| select with IdEst rather than RankMe | VICReg ResNet-50 · ImageNet, all hyperparameters | top-1 accuracy of selected model: 69.1 -> 65.5 | Hurt |
| select with IdEst rather than LiDAR | DINO ViT-S · ImageNet, all hyperparameters | top-1 accuracy of selected model: 65.5 -> 65.5 | No clear effect |
| select with IdEst rather than alpha-ReQ | DINO ResNet-50 · ImageNet, all hyperparameters | top-1 accuracy of selected model: 63.0 -> 67.6 | Improved |
| use dimMST (IdEst) instead of TwoNN and MLE for ID estimation | intrinsic dimension estimator · 1-d helix, 200 points | estimated ID | Improved |
| lower IdEst (lower intrinsic dimension) | SSL representations · ImageNet linear probe, 33 models | top-1 accuracy: rho = -0.74, tau = -0.55 | Improved |
| evaluate IdEst on a single reference dataset (ImageNet) for transfer | SSL representations · iNat-18, iNat-21, CIFAR-10, CIFAR-100 | top-1 accuracy: rho -0.81 to -0.83 | Improved |
| monitor IdEst during pretraining | DINO, VICReg, I-JEPA · ImageNet offline probing | IdEst vs accuracy | Improved |
| use IdEst in first 10 epochs of ViT training | DINO ViT-S · online probing | IdEst informativeness | Hurt |
| replace IdEst with RankMe as proxy | foundation models · ImageNet | Spearman rho with accuracy: -0.74 -> 0.49 (magnitude) | Hurt |
| replace linear probe with IdEst for evaluation | DINOv2 ViT-G · ImageNet-1k | wall-clock minutes: 322.7 -> 12.8 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- LiDAR — LiDAR: Sensing Linear Probing Performance in Joint Embedding SSL Architectures (ICLR 2024) · better on 4 of 4 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Position: An Empirically Grounded Identifiability Theory Will Accelerate Self Supervised Learning Research (ICML 2025 · similarity 0.88)
- LDReg: Local Dimensionality Regularized Self-Supervised Learning (ICLR 2024 · similarity 0.85)
- Preventing Dimensional Collapse in Self-Supervised Learning via Orthogonality Regularization (NeurIPS 2024 · similarity 0.85)
- T-REGS: Minimum Spanning Tree Regularization for Self-Supervised Learning (NeurIPS 2025 · similarity 0.85)
- Self-supervised Representation Learning from Random Data Projectors (ICLR 2024 · similarity 0.84)
- Collapse-Proof Non-Contrastive Self-Supervised Learning (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
