Research map / Architectures and efficiency / Pruning and efficient tuning
BWS: Best Window Selection Based on Sample Scores for Data Pruning across Broad Ranges
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Best Window Selection (BWS), which sorts samples by difficulty score, picks the best contiguous window via a kernel ridge regression proxy evaluated on the full training set, and reports strong data pruning across 1%-90% selection ratios on CIFAR-10/100 and ImageNet.
Problem
Existing data subset selection methods specialize either in high selection ratios (difficulty score-based) or low selection ratios (optimization-based), with no single method consistently competitive across the full range of selection ratios.
Approach
Sort samples by a difficulty score and search over contiguous windows of fixed size with varying starting points; rank windows cheaply by solving kernel ridge regression (conjugate-kernel proxy) on each window's features and evaluating accuracy on the full training set, avoiding training a network per window.
Key results
- On CIFAR-10 at 1-10% selection ratios, BWS gives a 15-30% test accuracy improvement over Forgetting, per the paper's claim.
- BWS reaches about 93% test accuracy on CIFAR-10 with only a 40% subset and matches the Oracle window closely (e.g., 91.80 vs 91.80 at 30%).
- BWS selection on CIFAR-10 with ResNet18 and 5% step size takes under 11 seconds, much cheaper than optimization-based baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BWS window selection rather than Forgetting-score selection | CIFAR-10 data pruning · CIFAR-10, 1% selection | test accuracy: 30.08 vs. 41.53 | Helped |
| Use BWS window selection rather than Forgetting-score selection | CIFAR-10 data pruning · CIFAR-10, 40% selection | test accuracy: 93.41 | Helped |
| Use one contiguous window rather than a wider window with random selection (ablation) | CIFAR-10 window subset selection · CIFAR-10, 40% selection | test accuracy: 92.62 -> 93.59 | Helped |
| Use one contiguous window rather than a union of two half-width windows (ablation) | CIFAR-10 window subset selection · CIFAR-10, 20% selection | test accuracy: 89.16 vs. 89.06 | No clear effect |
| Select windows using a smaller ConvNet proxy rather than KRR (ablation) | BWS proxy task · CIFAR-10, 1% selection | test accuracy: 46.25 vs. 46.10 | No clear effect |
| Select windows by minimizing average-gradient difference rather than KRR (ablation) | BWS proxy task · CIFAR-10, 1% selection | test accuracy: 39.45 vs. 46.10 | Hurt |
| Select windows by maximizing average-gradient cosine similarity rather than KRR (ablation) | BWS proxy task · CIFAR-10, 1% selection | test accuracy: 36.33 vs. 46.10 | Hurt |
Surprising findings
- Forgetting-based selection underperforms random selection at low ratios on CIFAR-10 (e.g., 30.08% vs 39.10% at 1%).
- Selecting the hardest samples alone yields poor feature spaces (tr(Sigma_W Sigma_B^+) of 9.33 for hardest 20% vs 1.04 for full data).
Limitations
- Relies on a fixed difficulty score (e.g., Forgetting) computed from full-dataset training, which itself is costly.
- Theoretical analysis uses a simplified linear binary classification setting with a wide gap between sample-deficient and sample-sufficient regimes.
- Robustness to label noise required a modified variant evaluating KRR only on the low-scoring 50% of samples.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet
Benchmarks: CIFAR-10 CIFAR-100 ImageNet
Metrics: test accuracy
Baselines: Random Forgetting EL2N AdaCore LCMat Moderate DS CCS SSL Prototype Memorization SVP
Builds on: Forgetting score Selection via proxy CRAIG LCMat
Closest papers by meaning
- A CLIP-Powered Framework for Robust and Generalizable Data Selection (ICLR 2025 · similarity 0.86)
- Diversified Batch Selection for Training Acceleration (ICML 2024 · similarity 0.85)
- Most Influential Subset Selection: Challenges, Promises, and Beyond (NeurIPS 2024 · similarity 0.85)
- Foundation Model Insights and a Multi-Model Approach for Superior Fine-Grained One-shot Subset Selection (ICML 2025 · similarity 0.84)
- DsDm: Model-Aware Dataset Selection with Datamodels (ICML 2024 · similarity 0.84)
- SAND: One-Shot Feature Selection with Additive Noise Distortion (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.