Research map / Label-efficient and robust learning / Recognition and long-tail
REDUCR: Robust Data Downsampling using Class Priority Reweighting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes REDUCR, an online batch selection method that uses multiplicative-weights class priority reweighting with amortised class-irreducible loss models to downsample training data while preserving worst-class test accuracy.
Problem
Data downsampling for streaming training often hurts worst-class generalization under class imbalance, noise, and distribution shift; existing online batch selection methods (e.g., RHO-Loss, train loss) optimize average performance.
Approach
Formulates robust data downsampling as a maximin objective over class-specific holdout likelihoods and solves it online via multiplicative weights over classes, with selection scores combining model loss, amortised class-irreducible loss (class-upweighted reference models), and clipped excess loss.
Key results
- On Clothing1M, REDUCR reaches 53.91% worst-class test accuracy vs 40.37% for Train Loss and 27.77% for RHO-Loss (5 runs).
- On CIFAR10, worst-class test accuracy 83.29% vs 78.80% for RHO-Loss; on CINIC10 75.30% vs 69.39%.
- Average test accuracy on CIFAR100 is 62.21% vs 60.95% for RHO-Loss, while CINIC10 average is slightly lower than RHO-Loss (81.68 vs 82.09).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use REDUCR instead of RHO-Loss | ResNet-18 · CIFAR10 | worst-class test accuracy: 78.80 ± 2.09 -> 83.29 ± 0.84 | Helped |
| Use REDUCR instead of RHO-Loss | ResNet-18 · CINIC10 | worst-class test accuracy: 69.39 ± 3.56 -> 75.30 ± 0.85 | Helped |
| Use REDUCR instead of Train Loss | ResNet-18 · CIFAR100 | worst-class test accuracy: 17.59 ± 5.17 -> 26.00 ± 2.65 | Helped |
| Use REDUCR instead of Train Loss | ImageNet-pretrained ResNet-18 · Clothing1M | worst-class test accuracy: 40.37 ± 3.58 -> 53.91 ± 2.42 | Helped |
| Use REDUCR instead of RHO-Loss | bert-base-uncased · MNLI | worst-class test accuracy: 76.74 ± 0.93 -> 79.45 ± 0.39 | Helped |
| Use REDUCR instead of Train Loss | bert-base-uncased · QQP | worst-class test accuracy: 79.96 ± 2.34 -> 86.61 ± 0.49 | Helped |
| Remove the model loss term from the selection rule (ablation) | REDUCR · CINIC10 | worst-class test accuracy | Hurt |
| Add the model loss term to the selection rule (ablation) | REDUCR · CINIC10 | worst-class test accuracy | Helped |
| Remove the amortised class-irreducible loss term (ablation) | REDUCR · CINIC10 | worst-class test accuracy | Hurt |
| Add the amortised class-irreducible loss term (ablation) | REDUCR · CINIC10 | worst-class test accuracy | Helped |
| Remove the class-holdout loss term (ablation) | REDUCR · CINIC10 | worst-class test accuracy | Hurt |
| Add the class-holdout loss term (ablation) | REDUCR · CINIC10 | worst-class test accuracy | Helped |
Surprising findings
- Under worst-class checkpointing, RHO-Loss's Clothing1M worst-class accuracy drops to 32.19%, while Train Loss improves to 49.02%, showing checkpoint choice strongly affects baseline rankings.
- A smaller percent-train (0.05) selecting from more candidates attained lower final worst-class and average accuracy, contrary to the authors' intuition.
Limitations
- Computational cost of selection scales linearly with the number of classes, requiring one class-irreducible model per class.
- Hyperparameter sensitivity (gamma, eta) analysed mainly on CIFAR10; sensitivity on other datasets needs more study.
- Worst-class accuracy results on Clothing1M are noisy with high variance across runs.
Experimental setup
Datasets: CIFAR10 CINIC10 CIFAR100 Clothing1M MNLI QQP
Benchmarks: CIFAR10 CINIC10 CIFAR100 Clothing1M MNLI QQP
Metrics: worst-class test accuracy average test accuracy
Baselines: Uniform Train Loss RHO-Loss Payoff (direct maximin selection)
Builds on: RHO-Loss multiplicative weights update Train Loss online batch selection (Katharopoulos & Fleuret)
Closest papers by meaning
- Restoring balance: principled under/oversampling of data for optimal classification (ICML 2024 · similarity 0.84)
- Improving Generalization via Meta-Learning on Hard Samples (CVPR 2024 · similarity 0.84)
- Revive Re-weighting in Imbalanced Learning by Density Ratio Estimation (NeurIPS 2024 · similarity 0.84)
- DRoP: Distributionally Robust Data Pruning (ICLR 2025 · similarity 0.83)
- Latent Score-Based Reweighting for Robust Classification on Imbalanced Tabular Data (ICML 2025 · similarity 0.83)
- Robust Visual Recognition with Class-Imbalanced Open-World Noisy Data (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.