Research map / Label-efficient and robust learning / Recognition and long-tail
Revive Re-weighting in Imbalanced Learning by Density Ratio Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Re-weighting with Density Ratio (RDR), which estimates per-class density ratios via first-order moment matching against a momentum-updated balanced feature prototype to dynamically set class weights during training for long-tailed classification.
Problem
Deep models trained on long-tailed, imbalanced data are biased toward frequent classes and generalize poorly to underrepresented classes; static re-weighting schemes fail to track how class-wise training dynamics change.
Approach
Dynamically computes sample weights as a density ratio between the imbalanced training feature distribution and a balanced distribution approximated by a per-class momentum feature prototype, obtained in closed form via first-order moment matching, and combines it with logit adjustment and SAM.
Key results
- On CIFAR-10-LT with IF=100, RDR+LA reaches 83.4% top-1 vs 82.2% for LA, and RDR+CE improves Few-class accuracy by 19.8% over CE.
- On ImageNet-LT, RDR+LA improves over LA from 47.9% to 48.1% overall, and with SAM the LA variant gains 7.1% per the ablation text.
- On Places-LT, RDR+CE raises overall accuracy from 29.3% to 39.4%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add RDR to LA | LA · CIFAR-10-LT IF=10 | top-1 accuracy: 89.2±0.3 -> 90.2±0.4 | Helped |
| Add RDR to LA | LA · CIFAR-10-LT IF=100 | top-1 accuracy: 82.2±0.7 -> 83.4±0.3 | Helped |
| Add RDR to LA | LA · CIFAR-100-LT IF=10 | top-1 accuracy: 62.3±0.5 -> 62.9±0.2 | Helped |
| Add RDR to LA | LA · CIFAR-100-LT IF=100 | top-1 accuracy: 48.2±0.4 -> 49.4±0.3 | Helped |
| Add RDR to LA | LA · ImageNet-LT | top-1 accuracy: 47.9±0.4 -> 48.1±0.3 | Helped |
| Add RDR to LA | LA · Places-LT | top-1 accuracy: 37.5±0.2 -> 39.5±0.1 | Helped |
| Add RDR to SAM with CE | SAM with CE · ImageNet-LT | top-1 accuracy: 45.8 -> 48.1 | Helped |
| Add RDR to SAM with CE | SAM with CE · Places-LT | top-1 accuracy: 38.2 -> 40.9 | Helped |
| Replace RDR with inverse-frequency weighting (1/n) under CE (ablation) | RDR method · CIFAR-10-LT IF=100 | top-1 accuracy: 82.88 -> 77.78 | Hurt |
| Replace RDR with inverse-frequency weighting (1/n) under LA (ablation) | RDR method · ImageNet-LT | top-1 accuracy: 48.12 -> 45.86 | Hurt |
| Replace RDR with inverse-frequency weighting (1/n) under CE (ablation) | RDR method · CIFAR-100-LT IF=100 | top-1 accuracy: 48.54 -> 44.21 | Hurt |
| Replace RDR with inverse-frequency weighting (1/n) under LA (ablation) | RDR method · Places-LT | top-1 accuracy: 40.77 -> 37.48 | Hurt |
Surprising findings
- Constant inverse-frequency re-weighting (1/n) is weaker than dynamic RDR weights, yet the paper argues static re-weighting falls behind other approaches largely because weights mismatch training dynamics.
Limitations
- Scalability concerns when the number of classes is very large (e.g., face recognition), due to matrix inversion cost.
- Risk of over-rebalancing toward minority classes that may hurt majority-class performance and fairness.
- Error bars reported only for CIFAR datasets due to computational constraints.
Experimental setup
Datasets: CIFAR-10-LT CIFAR-100-LT ImageNet-LT Places-LT CIFAR-10-LT-NL CIFAR-100-LT-NL
Benchmarks: CIFAR-10-LT CIFAR-100-LT ImageNet-LT Places-LT CIFAR-10-LT-NL CIFAR-100-LT-NL
Metrics: top-1 accuracy Many/Medium/Few class accuracy maximum and minimum Hessian eigenvalues Hessian trace
Baselines: CE Focal loss CB loss LA SAM ImbSAM CCSAM inverse frequency weighting
Builds on: class-balanced loss logit adjustment sharpness-aware minimization focal loss Wang et al. 2023 generalization bound for re-weighting
Closest papers by meaning
- Inverse Weight-Balancing for Deep Long-Tailed Learning (AAAI 2024 · similarity 0.87)
- Gradient Reweighting: Towards Imbalanced Class-Incremental Learning (CVPR 2024 · similarity 0.85)
- Overcoming Saturation in Density Ratio Estimation by Iterated Regularization (ICML 2024 · similarity 0.84)
- Deep Imbalanced Regression via Hierarchical Classification Adjustment (CVPR 2024 · similarity 0.84)
- Balancing the Scales: A Theoretical and Algorithmic Framework for Learning from Imbalanced Data (ICML 2025 · similarity 0.84)
- REDUCR: Robust Data Downsampling using Class Priority Reweighting (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.