Research map / Label-efficient and robust learning / Self-supervised and contrastive
LiDAR: Sensing Linear Probing Performance in Joint Embedding SSL Architectures
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LiDAR, an LDA-based effective-rank metric that uses the SSL objective's surrogate classes to label-free predict linear-probe accuracy and select hyperparameters for joint-embedding SSL methods, outperforming RankMe.
Problem
Evaluating representation quality of joint-embedding SSL models without downstream tasks or labels, for model selection and hyperparameter tuning; existing covariance-rank measures like RankMe fail to discriminate informative from uninformative directions.
Approach
Replace the raw feature covariance in RankMe with a whitened between-class (LDA) matrix, where clean images serve as surrogate classes and their augmented/masked views as samples, and compute the smooth effective rank of that LDA matrix.
Key results
- On I-JEPA grid search, LiDAR (student) reaches overall Kendall's tau 0.8159 vs 0.5830 for RankMe and Spearman 0.9494 vs 0.7700.
- On VICReg after 100 epochs, LiDAR achieves Spearman 0.9161 and Kendall's tau 0.8105 vs 0.3174 and 0.2056 for RankMe.
- On SimCLR random search, LiDAR recovers 58.93% probe accuracy vs ImageNet oracle 59.14% and RankMe 56.46%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LiDAR instead of RankMe for the representation-quality proxy | I-JEPA ViT-Base · ImageNet-1K grid search, overall hyperparameter ranking | Kendall's τ: 0.5830 -> 0.8159 | Helped |
| Use LiDAR instead of augmented RankMe for the representation-quality proxy | I-JEPA ViT-Base · ImageNet-1K grid search, overall hyperparameter ranking | Kendall's τ: 0.7513 -> 0.8159 | Helped |
| Evaluate the teacher branch rather than the student branch | LiDAR on I-JEPA ViT-Base · ImageNet-1K grid search, overall hyperparameter ranking | Kendall's τ: 0.8159 -> 0.6828 | Hurt |
| Use LiDAR instead of RankMe for the representation-quality proxy | data2vec ViT-Base · ImageNet-1K, overall hyperparameter ranking | Kendall's τ: 0.2238 -> 0.5531 | Helped |
| Use LiDAR instead of augmented RankMe for the representation-quality proxy | data2vec ViT-Base · ImageNet-1K, overall hyperparameter ranking | Kendall's τ: 0.2799 -> 0.5531 | Helped |
| Use LiDAR instead of RankMe for the representation-quality proxy | VICReg ResNet-50 · ImageNet-1K, epoch 100 | Spearman rank: 0.3174 -> 0.9161 | Helped |
| Use LiDAR instead of RankMe for the representation-quality proxy | VICReg ResNet-50 · ImageNet-1K, epoch 100 | Kendall's τ: 0.2056 -> 0.8105 | Helped |
| Use LiDAR instead of RankMe for the representation-quality proxy | DINO ViT-Small · ImageNet-1K hyperparameter ranking | Spearman rank: 0.8191 -> 0.8807 | Helped |
| Use LiDAR instead of RankMe for the representation-quality proxy | DINO ViT-Small · ImageNet-1K hyperparameter ranking | Kendall's τ: 0.6288 -> 0.7299 | Helped |
| Use LiDAR instead of RankMe for the representation-quality proxy | SimCLR ResNet-50 · ImageNet-1K grid search, overall hyperparameter ranking | Spearman rank: 0.5125 -> 0.9155 | Helped |
| Use LiDAR instead of augmented RankMe for the representation-quality proxy | SimCLR ResNet-50 · ImageNet-1K grid search, overall hyperparameter ranking | Kendall's τ: 0.5581 -> 0.7967 | Helped |
| Select hyperparameters using LiDAR rather than RankMe | data2vec ViT-Base · ImageNet-1K, overall hyperparameter selection | linear probe accuracy: 48.6980 -> 59.3720 | Helped |
Surprising findings
- LiDAR shows negative correlation with probe accuracy in some VICReg settings with high-dimensional embeddings.
- Augmented RankMe can match LiDAR for the best selected I-JEPA model, trailing only when no augmentation is used.
Limitations
- Negative correlation with probe accuracy observed for VICReg with high-dimensional embeddings.
- Extra computational cost from high-dimensional matrix inversion; not practical as a per-iteration training loss.
- Only linear probing evaluated; cross-method comparisons hard because positive-pair sampling differs across methods.
Experimental setup
Datasets: ImageNet-1K CIFAR10 CIFAR100 EuroSAT Food101 SUN397
Benchmarks: ImageNet-1K CIFAR10 CIFAR100 EuroSAT Food101 SUN397
Metrics: Spearman rank correlation Kendall's tau linear probe accuracy of top-ranked model
Baselines: RankMe RankMe (aug.) alpha-Req ImageNet oracle
Builds on: RankMe alpha-Req linear discriminant analysis
Closest papers by meaning
- Joint‑Embedding vs Reconstruction: Provable Benefits of Latent Space Prediction for Self‑Supervised Learning (NeurIPS 2025 · similarity 0.85)
- PSA-SSL: Pose and Size-aware Self-Supervised Learning on LiDAR Point Clouds (CVPR 2025 · similarity 0.84)
- LiT: Unifying LiDAR "Languages" with LiDAR Translator (NeurIPS 2024 · similarity 0.83)
- Learning Dynamic Evidence Routes for Vision Transformer Probing (NeurIPS 2026 · similarity 0.83)
- Drive vs. Decay: On the Training Dynamics of Joint-Embedding Predictive Architectures (NeurIPS 2026 · similarity 0.82)
- Rosetta Stone For Unified MLLMs: A Unified Tokenizer to Decipher Understanding and Generation (CVPR 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.