Research map / Learning theory and algorithms / Prediction and decision losses
Refined Coreset Selection: Towards Minimal Coreset Size under Model Performance Constraints
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes refined coreset selection (RCS), finding the smallest coreset whose trained-model performance stays within a tolerance, via lexicographic bilevel coreset selection (LBCS) using a randomized direct search.
Problem
Standard coreset selection fixes the coreset size in advance; practitioners instead want the smallest coreset that still matches full-data model performance, which is a multi-objective problem with priority between performance and size.
Approach
Formulates coreset selection as cardinality-free bilevel optimization with lexicographic priority (performance first with a compromise epsilon, then coreset size), optimized by a black-box randomized direct search (LexiFlow) that compares masks via lexicographic relations, with a convergence guarantee.
Key results
- On CIFAR-10 with predefined k=1000, LBCS reaches 48.3% test accuracy with an optimized coreset of ~970 examples, the best accuracy among compared methods.
- On Fashion-MNIST at k=2000, LBCS achieves 82.8% accuracy with a coreset of ~1915 examples versus 82.6% for Probabilistic at the same size budget.
- On ImageNet-1k at 70% ratio, LBCS reaches 89.98% top-5 accuracy using 68.53% of data, exceeding EL2N (89.82% at 70%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LBCS instead of the strongest fixed-size coreset baseline | Coreset selection · SVHN, k=1000–4000 | test accuracy (%): 70.3±0.8 vs. 70.6±0.3; 77.9±0.7 vs. 78.3±0.7; 81.9±0.6 vs. 82.3±0.7; 84.3±1.4 vs. 84.6±0.6 | Helped |
| Use LBCS instead of the strongest fixed-size coreset baseline | Coreset selection · F-MNIST, k=1000–4000 | test accuracy (%): 80.3±0.6 vs. 79.7±0.7; 82.6±0.2 vs. 82.8±0.6; 83.7±0.9 vs. 84.0±0.6; 84.2±0.7 vs. 84.5±0.4 | Mixed |
| Use LBCS instead of the strongest fixed-size coreset baseline | Coreset selection · CIFAR-10, k=1000–4000 | test accuracy (%): 48.2±0.9 vs. 48.3±1.2; 60.1±0.8 vs. 60.4±1.0; 69.2±2.6 vs. 69.5±0.9; 73.9±0.4 vs. 73.4±0.5 | Mixed |
| Increase search times from T=100 to T=500 (ablation) | LBCS · F-MNIST, k=1000 and 2000 | test accuracy (%): 77.0±1.8 -> 79.7±0.7; 80.2±1.9 -> 82.8±0.6 | Helped |
| Increase search times from T=500 to T=2000 (ablation) | LBCS · F-MNIST, k=1000 and 2000 | test accuracy (%): 79.7±0.7 -> 79.8±0.6; 82.8±0.6 -> 82.8±0.8 | No clear effect |
| Increase search times from T=100 to T=2000 (ablation) | LBCS · F-MNIST, k=1000 and 2000 | coreset size: 998.0±1.9 -> 935.8±3.8; 1995.6±2.5 -> 1893.9±4.3 | Helped |
Surprising findings
- Naively weighting coreset size and performance with lambda=1/2 causes the coreset size to be minimized too aggressively while performance stays poor, as shown by gradient-norm analysis.
- Allowing a small compromise epsilon in performance can improve generalization under label noise by reducing overfitting in coreset selection.
Limitations
- Theoretical convergence relies on strong assumptions (progressable and stable moving conditions) that are assumed rather than verified.
- ImageNet-1k experiments were run once per setting due to computational cost, and some baselines were not reported (Influential).
Experimental setup
Datasets: Fashion-MNIST SVHN CIFAR-10 ImageNet-1k MNIST
Benchmarks: Fashion-MNIST SVHN CIFAR-10 ImageNet-1k
Metrics: test accuracy coreset size
Baselines: Uniform sampling EL2N GraNd Influential coreset Moderate coreset CCS Probabilistic coreset
Builds on: Probabilistic bilevel coreset selection LexiFlow EL2N CCS
Closest papers by meaning
- Stable Coresets via Posterior Sampling: Aligning Induced and Full Loss Landscapes (NeurIPS 2025 · similarity 0.90)
- Feature Distribution Matching by Optimal Transport for Effective and Robust Coreset Selection (AAAI 2024 · similarity 0.88)
- Efficient Core-set Selection for Deep Learning Through Squared Loss Minimization (ICML 2025 · similarity 0.88)
- Mind the Boundary: Coreset Selection via Reconstructing the Decision Boundary (ICML 2024 · similarity 0.88)
- $\mathbb{D}^2$ Pruning: Message Passing for Balancing Diversity & Difficulty in Data Pruning (ICLR 2024 · similarity 0.87)
- STAFF: Speculative Coreset Selection for Task-Specific Fine-tuning (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.