Research map / Label-efficient and robust learning / Recognition and long-tail
Balancing Feature Similarity and Label Variability for Optimal Size-Aware One-shot Subset Selection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BOSS, a size-aware one-shot subset selection method that balances feature similarity (diversity) and EL2N-based label variability (difficulty) via a Beta-shaped importance function and facility-location submodular selection.
Problem
One-shot core-set selection for data-efficient deep learning typically favors either diverse or difficult samples independently of subset size, failing to represent the joint feature-label distribution.
Approach
A balanced core-set loss bound combines feature distance and label variability; EL2N difficulty is used as a surrogate lower bound for label variability, and a Beta importance function whose parameters depend on subset size and mean difficulty controls the diversity-difficulty trade-off in a facility-location objective optimized by lazy greedy.
Key results
- On Tiny-ImageNet at 10% subset, BOSS reaches 32.54% vs 29.59% for CCS (best baseline).
- On CIFAR100 at 10% subset, BOSS reaches 46.54% vs 40.26% for CCS.
- On ImageNet-1K at 10% subset, BOSS reaches 68.53% vs 59.10% for CCS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use BOSS instead of CCS | one-shot subset selection · TinyImageNet, 10% subset | accuracy: 29.59±0.9 -> 32.54±0.9 | Helped |
| use BOSS instead of CCS | one-shot subset selection · CIFAR-100, 10% subset | accuracy: 40.26±1.6 -> 46.54±0.9 | Helped |
| use BOSS instead of CCS | one-shot subset selection · CIFAR-10, 10% subset | accuracy: 74.78±1.8 -> 78.27±0.9 | Helped |
| use BOSS instead of CCS | one-shot subset selection · SVHN, 8% subset | accuracy: 86.69±1.5 -> 88.83±1.8 | Helped |
| add difficulty scoring to diversity-based selection (ablation) | BOSS on TinyImageNet · TinyImageNet, 10% subset | accuracy: 24.04±1.1 -> 31.23±0.9 | Helped |
| add diversity-based selection to difficulty-only selection (ablation) | BOSS on TinyImageNet · TinyImageNet, 10% subset | accuracy: 3.39±0.3 -> 31.23±0.9 | Helped |
| add cutoff to balanced diversity-and-difficulty selection (ablation) | BOSS on TinyImageNet · TinyImageNet, 10% subset | accuracy: 31.23±0.9 -> 32.54±0.9 | Helped |
| add cutoff to balanced diversity-and-difficulty selection (ablation) | BOSS on CIFAR-100 · CIFAR-100, 10% subset | accuracy: 47.41±0.6 -> 46.54±0.9 | Hurt |
| add cutoff to balanced diversity-and-difficulty selection (ablation) | BOSS on CIFAR-10 · CIFAR-10, 10% subset | accuracy: 76.19±2.3 -> 78.27±0.9 | Helped |
| add cutoff to balanced diversity-and-difficulty selection (ablation) | BOSS on SVHN · SVHN, 8% subset | accuracy: 85.86±1.8 -> 88.83±1.8 | Helped |
Surprising findings
- Pure difficulty-based selection (EL2N only) performs worst at small subset sizes, e.g. 7.11% on CIFAR100 at 10%.
Limitations
- Relies on Lipschitz-based theory with a heuristic surrogate; the Beta parameters (ca, cb) are tuned empirically per dataset.
- Requires an initial full-data training run (about 10 epochs) to compute EL2N scores.
- Cutoff mechanism needs full-epoch AUM training, adding cost.
Experimental setup
Datasets: CIFAR10 CIFAR100 SVHN Tiny-ImageNet ImageNet-1K ImageNet-10 synthetic four-moon dataset
Benchmarks: CIFAR10 CIFAR100 SVHN Tiny-ImageNet ImageNet-1K
Metrics: test accuracy
Baselines: Random CRAIG GradMatch Adacore LCMAT Moderate CCS YOCO
Builds on: core-set loss bound (Sener & Savarese) EL2N score CCS facility location submodular selection
Closest papers by meaning
- Foundation Model Insights and a Multi-Model Approach for Superior Fine-Grained One-shot Subset Selection (ICML 2025 · similarity 0.88)
- Efficient Core-set Selection for Deep Learning Through Squared Loss Minimization (ICML 2025 · similarity 0.87)
- Contributing Dimension Structure of Deep Feature for Coreset Selection (AAAI 2024 · similarity 0.87)
- Multinoulli Extension: A Lossless Yet Effective Probabilistic Framework for Subset Selection over Partition Constraints (ICML 2025 · similarity 0.85)
- SAND: One-Shot Feature Selection with Additive Noise Distortion (ICML 2025 · similarity 0.85)
- Going Beyond Feature Similarity: Effective Dataset distillation based on Class-aware Conditional Mutual Information (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.