Research map / Learning theory and algorithms / Prediction and decision losses
Balancing the Scales: A Theoretical and Algorithmic Framework for Learning from Imbalanced Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IMMAX, a class-imbalanced margin loss with per-class confidence margins that is H-consistent for the zero-one loss, with margin bounds and a logistic-style algorithm validated on long-tailed and step-imbalanced CIFAR/Tiny ImageNet.
Problem
Class imbalance in binary and multi-class classification lacks theoretical foundations; existing cost-sensitive and logit-modification methods are not Bayes-consistent for the standard zero-one loss, and prior margin analysis targets the balanced loss.
Approach
Introduce class-imbalanced (rho+, rho-) and multi-class rho-margin losses that apply distinct confidence margins per class, prove H-consistency for the zero-one loss, derive class-sensitive Rademacher margin bounds, and derive the IMMAX algorithm (a class-dependent logistic loss) from them.
Key results
- On long-tailed CIFAR-10 at imbalance ratio 200, IMMAX reaches 96.11% accuracy vs 95.45% for LDAM (best baseline) with ResNet-34.
- On long-tailed CIFAR-100 at ratio 200, IMMAX reaches 80.47% vs 79.53% for focal loss.
- IMMAX outperforms baselines on all long-tailed and step-imbalanced settings with ratios 200, 100, and 10, with at least 0.6% absolute gain over the runner-up.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use IMMAX instead of the strongest listed baseline | ResNet-34 · CIFAR-10 long-tailed, ratio 200 | accuracy: 95.45 $\pm$ 0.38 -> 96.11 $\pm$ 0.34 | Helped |
| Use IMMAX instead of the strongest listed baseline | ResNet-34 · CIFAR-100 long-tailed, ratio 200 | accuracy: 79.53 $\pm$ 0.75 -> 80.47 $\pm$ 0.68 | Helped |
| Use IMMAX instead of the strongest listed baseline | ResNet-34 · Tiny ImageNet long-tailed, ratio 200 | accuracy: 63.70 $\pm$ 0.62 -> 65.20 $\pm$ 0.65 | Helped |
| Use IMMAX instead of the strongest listed baseline | ResNet-34 · CIFAR-10 step-imbalanced, ratio 200 | accuracy: 94.85 $\pm$ 0.18 -> 95.42 $\pm$ 0.30 | Helped |
| Use IMMAX instead of the strongest listed baseline | ResNet-34 · CIFAR-100 step-imbalanced, ratio 200 | accuracy: 77.18 $\pm$ 0.50 -> 78.21 $\pm$ 0.48 | Helped |
| Use IMMAX instead of the strongest listed baseline | ResNet-34 · Tiny ImageNet step-imbalanced, ratio 200 | accuracy: 62.54 $\pm$ 0.51 -> 63.57 $\pm$ 0.36 | Helped |
| Use IMMAX instead of hinge loss | Linear model · CIFAR-10 binarized, Airplane vs. all | accuracy: 90.17 $\pm$ 0.09 -> 91.02 $\pm$ 0.06 | Helped |
| Use IMMAX instead of hinge loss | Linear model · CIFAR-10 binarized, Automobile vs. all | accuracy: 91.01 $\pm$ 0.13 -> 91.26 $\pm$ 0.05 | Helped |
| Use IMMAX instead of hinge loss | Linear model · CIFAR-10 binarized, Horse vs. all | accuracy: 90.58 $\pm$ 0.11 -> 91.03 $\pm$ 0.03 | Helped |
| Assign shared margin parameters to minority classes and distinct parameters to frequent classes (ablation) | IMMAX · CIFAR-10, CIFAR-100, and Tiny ImageNet step-imbalanced settings | accuracy | No clear effect |
Surprising findings
- Cost-sensitive losses are shown not to be Bayes-consistent for the zero-one loss when class weights differ.
- The margin prescription of a prior analysis for balanced loss (positive margin smaller when m+>m-) is the opposite of what the zero-one-loss theory recommends.
Limitations
- Focus is theoretical; experiments limited to CIFAR and Tiny ImageNet with ResNet-34 and standard accuracy.
- Hyperparameters (rho_k) require cross-validation search around theoretical values.
- Bounds include large-scale and class-count dependence; not evaluated on large-scale benchmarks.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny ImageNet
Benchmarks: CIFAR-10 CIFAR-100 Tiny ImageNet
Metrics: accuracy
Baselines: cross-entropy re-weighting balanced softmax equalization loss logit adjusted loss class-balanced loss focal loss LDAM
Builds on: LDAM logit adjustment balanced softmax cost-sensitive learning
Closest papers by meaning
- Improved Balanced Classification with Theoretically Grounded Loss Functions (NeurIPS 2025 · similarity 0.89)
- Difficulty-aware Balancing Margin Loss for Long-tailed Recognition (AAAI 2025 · similarity 0.88)
- Restoring balance: principled under/oversampling of data for optimal classification (ICML 2024 · similarity 0.88)
- Balanced Data, Imbalanced Spectra: Unveiling Class Disparities with Spectral Imbalance (ICML 2024 · similarity 0.87)
- Learning under Localized Minority Imbalance (NeurIPS 2026 · similarity 0.87)
- Inverse Weight-Balancing for Deep Long-Tailed Learning (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.