Research map / Label-efficient and robust learning / Self-supervised and contrastive
Weighted Ensemble Self-Supervised Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add an entropy-weighted ensemble of projection heads and codebooks | DINO-P and MSN-P · ImageNet-1K, few-shot evaluation across ViT-S/16, ViT-B/16, and ViT-B/8 | 1-shot accuracy: up to 5.5 pp | Helped |
| Use entropy weighting rather than uniform weighting (ablation) | DINO-P projection-head and codebook ensemble · ImageNet-1K, ViT-S/16, 1-shot | accuracy: 40.6 -> 44.0 | Helped |
| Ensemble both projection head and codebook rather than projection head only (ablation) | DINO-P with probability weighting · ImageNet-1K, ViT-S/16, 1-shot | accuracy: 39.8 -> 41.9 | Helped |
| Use entropy weighting with projection head only rather than both head and codebook (ablation) | DINO-P ensemble · ImageNet-1K, ViT-S/16, 1-shot | accuracy: 44.0 -> 43.0 | Hurt |
| Use uniform weighting rather than entropy weighting (ablation) | DINO-P projection-head and codebook ensemble · ImageNet-1K, ViT-S/16, 1-shot | accuracy: 40.6 -> 40.4 | No clear effect |
| Weight ensemble heads by student entropy instead of teacher entropy (ablation) | DINO-P projection-head and codebook ensemble · ImageNet-1K, ViT-S/16, 1-shot | accuracy: 40.6 -> 40.0 | Hurt |
| Increase the number of EW ensemble heads up to 16 (ablation) | DINO-P · ImageNet-1K evaluation | accuracy | Helped |
| Use only 2 EW ensemble heads rather than a single head (ablation) | MSN-P · ImageNet-1K, ViT-S/16, 1-shot | accuracy: 47.4 -> 48.8 | Helped |
| Use an overly high or low entropy-weighting temperature (ablation) | EW ensemble · ImageNet-1K few-shot evaluation | accuracy | Hurt |
| Replace centering with Sinkhorn-Knopp normalization (ablation) | DINO · ImageNet-1K few-shot evaluation | accuracy: 2~6 percentage point gains for 1 shot | Helped |
| Reduce the projection-head hidden dimension from 2048 to 1024 (ablation) | DINO · ImageNet-1K few-shot evaluation | accuracy: 38.9 -> 44.6 | Helped |
| Increase the capacity of a single non-ensembled projection head too much (ablation) | DINO-P · ImageNet-1K few-shot evaluation | accuracy | Hurt |
Closest papers by meaning
- Temporal Ensembling for Semi-Supervised Learning (ICLR 2017 · similarity 0.87)
- EnsLoss: Stochastic Calibrated Loss Ensembles for Preventing Overfitting in Classification (ICML 2025 · similarity 0.86)
- Understanding Overadaptation in Supervised Fine-Tuning: The Role of Ensemble Methods (ICML 2025 · similarity 0.86)
- Forming Auxiliary High-confident Instance-level Loss to Promote Learning from Label Proportions (CVPR 2025 · similarity 0.85)
- Subsampled Ensemble Can Improve Generalization Tail Exponentially (NeurIPS 2025 · similarity 0.85)
- Improving Generalization via Meta-Learning on Hard Samples (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.