Research map / Label-efficient and robust learning / Recognition and long-tail
Improved Balanced Classification with Theoretically Grounded Loss Functions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Generalized Logit-Adjusted (GLA) and Generalized Class-Aware (GCA) surrogate losses for balanced multi-class classification, with H-consistency analysis and experiments on CIFAR-10/100 and Tiny ImageNet under long-tailed and step imbalance.
Problem
Directly minimizing the balanced classification loss under class imbalance is intractable, so effective tractable surrogate losses with consistency guarantees are needed.
Approach
Extends logit-adjusted losses to the general cross-entropy (GCE) family (GLA) and extends class-weighted losses with class-dependent confidence margins across the GCE family (GCA); proves GLA is Bayes-consistent and H-consistent only for complete hypothesis sets with 1/p_min scaling, while GCA is H-consistent for bounded or complete sets with 1/sqrt(p_min) scaling.
Key results
- GCA achieves 33.17% balanced error on CIFAR-100 long-tailed rho=1000 vs 38.45% for CE and 35.92% for LA (tau=1)
- GLA achieves best balanced error at rho=100 on CIFAR-10 (2.07%) and CIFAR-100 (53.68%) long-tailed settings
- GCA edges out GLA in the highly imbalanced rho=1000 setting in the long-tailed experiments
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace LA (τ=1) with GCA | ResNet-32 · Long-tailed CIFAR-10 (ρ=1000) | balanced error: 2.18 $\pm$ 0.18 -> 2.02 $\pm$ 0.15 | Helped |
| Replace LA (τ=1) with GCA | ResNet-32 · Long-tailed CIFAR-100 (ρ=1000) | balanced error: 35.92 $\pm$ 0.47 -> 33.17 $\pm$ 0.57 | Helped |
| Replace LA (τ=1) with GCA | ResNet-32 · Long-tailed Tiny ImageNet (ρ=1000) | balanced error: 67.17 $\pm$ 0.49 -> 64.88 $\pm$ 0.66 | Helped |
| Replace LA (τ=1) with GLA | ResNet-32 · Long-tailed CIFAR-10 (ρ=100) | balanced error: 2.23 $\pm$ 0.08 -> 2.07 $\pm$ 0.06 | Helped |
| Replace LA (τ=1) with GLA | ResNet-32 · Long-tailed CIFAR-100 (ρ=100) | balanced error: 56.23 $\pm$ 0.21 -> 53.68 $\pm$ 0.76 | Helped |
| Replace LA (τ=1) with GLA | ResNet-32 · Long-tailed Tiny ImageNet (ρ=100) | balanced error: 102.81 $\pm$ 0.89 -> 100.70 $\pm$ 0.83 | Helped |
| Replace LA (τ=1) with GCA | ResNet-32 · Step-imbalanced CIFAR-10 (ρ=1000) | balanced error: 5.54 $\pm$ 0.48 -> 5.35 $\pm$ 0.02 | Helped |
| Replace LA (τ=1) with GCA | ResNet-32 · Step-imbalanced CIFAR-100 (ρ=1000) | balanced error: 11.42 $\pm$ 0.33 -> 10.43 $\pm$ 0.15 | Helped |
| Replace LA (τ=1) with GCA | ResNet-32 · Step-imbalanced Tiny ImageNet (ρ=1000) | balanced error: 37.44 $\pm$ 0.25 -> 36.32 $\pm$ 0.32 | Helped |
| Replace LA (τ=1) with GLA | ResNet-32 · Step-imbalanced CIFAR-10 (ρ=100) | balanced error: 2.70 $\pm$ 0.12 -> 2.48 $\pm$ 0.11 | Helped |
| Replace LA (τ=1) with GLA | ResNet-32 · Step-imbalanced CIFAR-100 (ρ=100) | balanced error: 55.43 $\pm$ 0.63 -> 52.70 $\pm$ 0.15 | Helped |
| Replace LA (τ=1) with GLA | ResNet-32 · Step-imbalanced Tiny ImageNet (ρ=100) | balanced error: 38.42 $\pm$ 0.14 -> 36.71 $\pm$ 0.33 | Helped |
Surprising findings
- LA loss with tau != 1 is not Bayes-consistent for the balanced loss
- Even with tau=1, LA loss is not H-consistent for certain bounded hypothesis sets
Limitations
- Bounded hypothesis sets do not guarantee H-consistency for GLA losses
- GLA bounds scale as 1/p_min, less favorable under high imbalance
- Additional hyperparameters (q, rho_k) need tuning
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny ImageNet
Benchmarks: CIFAR-10 CIFAR-100 Tiny ImageNet
Metrics: balanced error
Baselines: CE WCE LA loss Equalization loss Class-Balanced loss Focal loss LDAM loss
Builds on: logit-adjusted loss general cross-entropy loss class-weighted cross-entropy h-consistency bounds
Closest papers by meaning
- Balancing the Scales: A Theoretical and Algorithmic Framework for Learning from Imbalanced Data (ICML 2025 · similarity 0.89)
- Class and Attribute-Aware Logit Adjustment for Generalized Long-Tail Learning (AAAI 2025 · similarity 0.87)
- Neural Collapse for Cross-entropy Class-Imbalanced Learning with Unconstrained ReLU Features Model (ICML 2024 · similarity 0.86)
- Trading off Consistency and Dimensionality of Convex Surrogates for Multiclass Classification (NeurIPS 2024 · similarity 0.86)
- Principled Algorithms for Optimizing Generalized Metrics in Binary Classification (ICML 2025 · similarity 0.85)
- Neural Collapse To Multiple Centers For Imbalanced Data (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.