Research map / Label-efficient and robust learning / Recognition and long-tail
Mind the Boundary: Coreset Selection via Reconstructing the Decision Boundary
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BoundarySet, a coreset selection method that picks training samples closest to the decision boundary (distance estimated via PGD step counts) plus a coverage-centric sampling variant, with a theory bounding generalization gap by decision boundary reconstruction error.
Problem
Coreset selection for efficient training: choosing a small training subset whose trained model generalizes like one trained on the full data, with heuristic geometry/score methods lacking generalization guarantees and optimization methods not scaling.
Approach
Select samples near the decision boundary, where distance is approximated by the number of PGD steps needed to flip the prediction, and combine with coverage-centric sampling; theoretically the decision boundary reconstruction error upper-bounds the generalization gap between full-data and coreset-trained models.
Key results
- On ImageNet-1K, BoundarySet-CCS at 50% pruning keeps accuracy within 1% of full-data training (ResNet-34 ~73.5% at 30% pruning with 0.1% drop).
- On CIFAR-10, BoundarySet-CCS shows zero accuracy decrease at 50% pruning and 87.9% accuracy at 90% pruning.
- Cross-architecture consistency of selected coresets (intersection ratio) stays above 0.5 across selection ratios, and ResNet-50 coreset training ConvNeXt drops less than 1% on ImageNet-1K at 50%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add coverage-centric sampling (CCS) to BoundarySet (ablation) | BoundarySet · ImageNet-1K, 70% pruning | accuracy: +0.4% | Helped |
| Add coverage-centric sampling (CCS) to BoundarySet (ablation) | BoundarySet · ImageNet-1K, 80% pruning | accuracy: +2.6% | Helped |
| Add coverage-centric sampling (CCS) to BoundarySet (ablation) | BoundarySet · ImageNet-1K, 90% pruning | accuracy: +7.7% | Helped |
| Add coverage-centric sampling (CCS) to BoundarySet (ablation) | BoundarySet · CIFAR-100, 50% pruning | accuracy: 75.5% | Helped |
| Use BoundarySet instead of CCS | Coreset selection on CIFAR-100 · CIFAR-100, 30% pruning | accuracy: nearly 1% | Helped |
| Use BoundarySet-CCS instead of BoundarySet (ablation) | Coreset selection on CIFAR-10 · CIFAR-10, 90% pruning | accuracy: 87.9% | Helped |
| Use a coreset selected by ResNet-50 to train ConvNeXt | BoundarySet-CCS · ImageNet-1K, 50% selection | accuracy: less than a 1% drop | Hurt |
| Use a coreset selected by Swin-Transformer to train ConvNeXt | BoundarySet-CCS · ImageNet-1K, 50% selection | accuracy: 1.36% decrease | Hurt |
Surprising findings
- BoundarySet alone performs poorly at high pruning ratios because boundary-only samples lose distribution coverage, which coverage-centric sampling fixes.
- Coresets selected with one architecture transfer to other architectures, including across families (CNN to ViT), with only modest accuracy drops.
Limitations
- Relies on the assumption of a good feature representation, which is hard to obtain with very sparse data at high pruning rates.
- Performance degrades at high pruning rates for the pure distance-based BoundarySet variant.
- Theory bounds the gap via a decision boundary reconstruction error that is estimated using synthetic BigGAN images rather than the true distribution.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet-1K
Benchmarks: CIFAR-10 CIFAR-100 ImageNet-1K
Metrics: test accuracy decision boundary reconstruction error
Baselines: Random Forgetting EL2N Moderate AUM CCS
Builds on: Coverage-centric coreset selection Projected Gradient Descent adversarial attack Support vector machines
Closest papers by meaning
- Stable Coresets via Posterior Sampling: Aligning Induced and Full Loss Landscapes (NeurIPS 2025 · similarity 0.88)
- Refined Coreset Selection: Towards Minimal Coreset Size under Model Performance Constraints (ICML 2024 · similarity 0.88)
- Pruning Neural Networks via Coresets and Convex Geometry: Towards No Assumptions (NeurIPS 2022 · similarity 0.87)
- $\mathbb{D}^2$ Pruning: Message Passing for Balancing Diversity & Difficulty in Data Pruning (ICLR 2024 · similarity 0.85)
- Feature Distribution Matching by Optimal Transport for Effective and Robust Coreset Selection (AAAI 2024 · similarity 0.85)
- Contributing Dimension Structure of Deep Feature for Coreset Selection (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.