Research map / Label-efficient and robust learning / Recognition and long-tail
ELFS: Label-Free Coreset Selection with Proxy Training Dynamics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ELFS, a label-free coreset selection method that estimates AUM training-dynamics difficulty using TEMI deep-clustering pseudo-labels and applies double-end pruning to correct the resulting score distribution shift.
Problem
Coreset selection normally relies on training-dynamics difficulty scores that need ground-truth labels, while existing label-free methods use geometry-based scores that often underperform random sampling.
Approach
Replace ground-truth labels with TEMI deep-clustering pseudo-labels to compute AUM training-dynamics scores, and use double-end pruning (prune a fraction of hardest examples, then easiest ones) to counter the easy-example bias induced by noisy pseudo-labels; the hard-pruning rate is tuned on pseudo-label validation accuracy.
Key results
- With SwAV encoder, ELFS outperforms D2 by up to 10.2% accuracy on ImageNet-1K.
- With DINO encoder, ELFS outperforms FreeSel by up to 3.9% accuracy on ImageNet-1K.
- ELFS is reported as the first label-free approach to outperform random sampling at all pruning rates on the evaluated benchmarks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ELFS instead of D2 with SwAV embeddings | Label-free coreset selection · ImageNet-1K, 90% pruning | accuracy: 43.2 -> 53.4 | Helped |
| Use ELFS instead of FreeSel with DINO embeddings | Label-free coreset selection · ImageNet-1K, 90% pruning | accuracy: 51.1 -> 54.9 | Helped |
| Use SCAN instead of TEMI for pseudo-label generation (ablation) | ELFS with DINO embeddings · CIFAR100, pruning rates 30%–90% | accuracy: 76.8 -> 75.9 (30%); 54.9 -> 53.7 (90%) | Mixed |
| Use K-Means instead of TEMI for pseudo-label generation (ablation) | ELFS with DINO embeddings · CIFAR100, pruning rates 30%–90% | accuracy: 76.8 -> 75.2 (30%); 54.9 -> 29.0 (90%) | Hurt |
| Add double-end pruning rather than using CCS sampling (ablation) | ELFS with pseudo-label AUM · CIFAR100, pruning rates 30%–90% | accuracy: 75.5 -> 76.8 (30%); 51.8 -> 54.9 (90%) | Helped |
| Add double-end pruning rather than using D2 sampling (ablation) | ELFS with pseudo-label AUM · CIFAR100, pruning rates 30%–90% | accuracy: 74.8 -> 76.8 (30%); 27.7 -> 54.9 (90%) | Helped |
| Use forgetting instead of AUM difficulty scores (ablation) | ELFS · CIFAR10 and CIFAR100, all reported pruning rates | accuracy: 95.5 vs 95.4 (CIFAR10, 30%); 76.8 vs 76.8 (CIFAR100, 30%) | No clear effect |
| Use EL2N instead of AUM difficulty scores (ablation) | ELFS · CIFAR10 and CIFAR100, all reported pruning rates | accuracy: 95.5 vs 95.6 (CIFAR10, 30%); 76.8 vs 76.5 (CIFAR100, 30%) | No clear effect |
| Replace pretrained DINO with a self-trained encoder (ablation) | ELFS · CIFAR10, 30%–90% pruning | accuracy: 95.5 -> 94.8 (30%); 87.3 -> 82.8 (90%) | Hurt |
| Use CLIP instead of DINO as encoder (ablation) | ELFS · CIFAR10 and CIFAR100, 30%–90% pruning | accuracy: 95.5 -> 95.0 (CIFAR10, 30%); 54.9 -> 50.6 (CIFAR100, 90%) | Hurt |
| Use SwAV instead of DINO as encoder (ablation) | ELFS · CIFAR10 and CIFAR100, 30%–90% pruning | accuracy: 95.5 -> 95.0 (CIFAR10, 30%); 54.9 -> 49.8 (CIFAR100, 90%) | Mixed |
| Select hard-pruning rate using pseudo-label validation rather than ground-truth labels (ablation) | ELFS · CIFAR100, 50% pruning | accuracy: 0.1% lower | Hurt |
Surprising findings
- Simply applying CCS on pseudo-label AUM scores already outperforms prior SOTA label-free methods, despite pseudo-labels being noisy.
- Pseudo-label accuracy on ImageNet-1K is below 60% even with DINO, yet coreset selection remains effective.
Limitations
- Hard pruning rate beta requires a grid search, adding overhead (about 17 hours on ImageNet-1K per pruning rate).
- Relies on strong pretrained encoders (DINO/SwAV); self-trained encoders reduce performance.
- Evaluated only on image classification benchmarks with ResNet architectures.
Experimental setup
Datasets: CIFAR10 CIFAR100 STL10 ImageNet-1K
Benchmarks: CIFAR10 CIFAR100 STL10 ImageNet-1K
Metrics: test accuracy
Baselines: Random Prototypicality D2 (label-free) FreeSel BADGE CCS (supervised)
Builds on: TEMI CCS D2 AUM
Closest papers by meaning
- Refined Coreset Selection: Towards Minimal Coreset Size under Model Performance Constraints (ICML 2024 · similarity 0.85)
- Label-Efficient Dataset Pruning via Semi-Supervised Pseudo-Labeling (NeurIPS 2026 · similarity 0.84)
- $\mathbb{D}^2$ Pruning: Message Passing for Balancing Diversity & Difficulty in Data Pruning (ICLR 2024 · similarity 0.84)
- Stable Coresets via Posterior Sampling: Aligning Induced and Full Loss Landscapes (NeurIPS 2025 · similarity 0.84)
- Beyond Random Augmentations: Pretraining with Hard Views (ICLR 2025 · similarity 0.84)
- Balancing Feature Similarity and Label Variability for Optimal Size-Aware One-shot Subset Selection (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.