Research map / Label-efficient and robust learning / Recognition and long-tail
DRoP: Distributionally Robust Data Pruning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that standard data pruning methods worsen class-wise bias and proposes DRoP, which randomly prunes within classes using class-wise quotas derived from validation error rates of a preliminary model.
Problem
Data pruning is evaluated only on average accuracy; its effect on classification bias (worst-class accuracy) of trained models is unknown, and existing pruning methods can produce highly biased classifiers.
Approach
Replace sample-level difficulty scores with random subsampling within classes, where per-class retention densities are set proportional to (1 - validation recall) from a preliminary model trained on the full data; a Gaussian-mixture analysis motivates worst-class optimal class priors.
Key results
- On CIFAR-100 with VGG-19 at 50% density, Random+DRoP raises worst-class accuracy from 35.8% (full dataset) to 45.4% at under 6% average accuracy cost.
- Random+DRoP improves all robustness metrics over the full dataset on all model-dataset pairs, while existing pruning methods (EL2N, GraNd, Forgetting, Dynamic Uncertainty) show little or negative worst-class gains.
- On imbalanced TinyImageNet, Random+DRoP consistently beats EL2N in average and worst-class accuracy across imbalance factors 2, 5, and 20.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use random pruning with DRoP class quotas instead of plain random pruning | VGG-19 · CIFAR-100, density 0.5 | worst-class accuracy (%): 30.1 -> 45.3 | Helped |
| use random pruning with DRoP class quotas instead of training on the full dataset | VGG-19 · CIFAR-100, density 0.5 | worst-class accuracy (%): 35.8 -> 45.3 | Helped |
| use random pruning with DRoP class quotas instead of plain random pruning | VGG-16 · CIFAR-10, density 0.5 | worst-class accuracy (%): 80.1 -> 85.9 | Helped |
| use random pruning with DRoP class quotas instead of plain random pruning | ResNet-18 · TinyImageNet, density 0.5 | worst-class accuracy (%): 10.0 -> 20.0 | Helped |
| use random pruning with DRoP class quotas instead of plain random pruning | VGG-19 · CIFAR-100, density 0.5 | average accuracy (%): 64.8 -> 63.7 | Hurt |
| add DRoP class quotas to Random pruning (ablation) | Random pruning · CIFAR-10, density 0.3 | worst-class accuracy (%): 75.5 -> 82.1 | Helped |
| add DRoP class quotas to Random pruning (ablation) | Random pruning · TinyImageNet, density 0.3 | worst-class accuracy (%): 8.7 -> 16.2 | Helped |
| add DRoP class quotas to Forgetting pruning (ablation) | Forgetting · CIFAR-100, density 0.3 | worst-class accuracy (%): 26.3 -> 32.0 | Helped |
| add DRoP class quotas to Forgetting pruning (ablation) | Forgetting · TinyImageNet, density 0.3 | worst-class accuracy (%): 1.0 -> 1.3 | No clear effect |
| add DRoP class quotas to Forgetting pruning (ablation) | Forgetting · TinyImageNet, density 0.5 | worst-class accuracy (%): 9.5 -> 12.7 | Helped |
Surprising findings
- Dynamic Uncertainty achieves strong average accuracy on CIFAR-100 and ImageNet yet fails badly on worst-class accuracy by removing entire difficult classes at 10% density.
- Random pruning according to class ratios from existing methods (Random+StrategyQ) outperforms the vanilla pruning algorithms on robustness, suggesting random within-class selection matters.
Limitations
- Focuses on classification bias; group-wise robustness and spurious correlations only lightly explored (Waterbirds).
- Limited cross-evaluation against the broad range of distributionally robust optimization methods.
- Theory is restricted to a toy one-dimensional mixture of two Gaussians with linear classifiers; error-based quotas lack formal guarantees.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 TinyImageNet ImageNet Waterbirds
Benchmarks: CIFAR-10 CIFAR-100 TinyImageNet ImageNet Waterbirds
Metrics: average accuracy worst-class accuracy max-min recall difference std of recalls worst-group accuracy
Baselines: Random EL2N GraNd Forgetting Dynamic Uncertainty CoreSet CDB-W Just Train Twice Learning from Failure Learning to Split Bias Amplification
Builds on: EL2N GraNd Forgetting Dynamic Uncertainty CDB-W
Closest papers by meaning
- Lightweight Dataset Pruning without Full Training via Example Difficulty and Prediction Uncertainty (ICML 2025 · similarity 0.89)
- Exploring Learning Complexity for Efficient Downstream Dataset Pruning (ICLR 2025 · similarity 0.87)
- Adaptive Sharpness-Aware Pruning for Robust Sparse Networks (ICLR 2024 · similarity 0.87)
- $\mathbb{D}^2$ Pruning: Message Passing for Balancing Diversity & Difficulty in Data Pruning (ICLR 2024 · similarity 0.86)
- Distilling the Knowledge in Data Pruning (ICML 2025 · similarity 0.86)
- Geometric Median (GM) Matching for Robust k-Subset Selection from Noisy Data (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.