Research map / Label-efficient and robust learning / Recognition and long-tail
Long-Tailed Learning as Multi-Objective Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Formulates long-tailed recognition as multi-objective optimization and proposes Gradient-Balancing Grouping (GBG), which clusters classes by gradient similarity and applies MGDA-style min-norm weighting over group gradients to get approximate Pareto descent updates.
Problem
Long-tailed classification suffers from a seesaw dilemma where rebalancing that helps tail classes hurts head classes, attributed to gradient imbalance between head and tail classes.
Approach
Treat each class (grouped by gradient similarity into a few groups) as an objective in a multi-objective optimization problem, and solve the min-norm problem over group gradients so each update is an approximate Pareto descent direction, with a group-aware sampler ensuring every batch covers all groups.
Key results
- GBG achieves 85.05% on CIFAR10-LT (IF=100) and 52.31% on CIFAR100-LT (IF=100), above BCL at 84.07% and 51.52%.
- On ImageNet-LT, GBG reaches 57.6% top-1 with ResNet-50 (vs. BCL 56.4%) and 58.7% with ResNeXt-50.
- On iNaturalist 2018, GBG reaches 71.9% top-1, versus 71.5% for BCL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply GBG instead of BCL | ResNet-32 · CIFAR10-LT IF=100 | top-1 accuracy: 84.07 -> 85.05 | Helped |
| apply GBG instead of BCL | ResNet-32 · CIFAR10-LT IF=50 | top-1 accuracy: 86.98 -> 87.73 | Helped |
| apply GBG instead of BCL | ResNet-32 · CIFAR100-LT IF=100 | top-1 accuracy: 51.52 -> 52.31 | Helped |
| apply GBG instead of BCL | ResNet-32 · CIFAR100-LT IF=50 | top-1 accuracy: 56.23 -> 57.18 | Helped |
| apply GBG instead of BCL | ResNet-50 · ImageNet-LT | top-1 accuracy: 56.4 -> 57.6 | Helped |
| apply GBG instead of BCL | ResNeXT-50 · ImageNet-LT | top-1 accuracy: 56.9 -> 58.7 | Helped |
| apply GBG instead of BCL | ResNet-50 · iNaturalist 2018 | top-1 accuracy: 71.5 -> 71.9 | Helped |
| replace gradient-based grouping with random grouping (ablation) | GBG, ResNet-50 · ImageNet-LT | top-1 accuracy: 57.49±0.29 -> 56.37±0.41 | Hurt |
| replace gradient-based grouping with instance-number grouping (ablation) | GBG, ResNet-50 · ImageNet-LT | top-1 accuracy: 57.49±0.29 -> 56.61±0.27 | Hurt |
| replace gradient-based grouping with random grouping (ablation) | GBG, ResNeXT-50 · ImageNet-LT | top-1 accuracy: 58.61±0.37 -> 56.96±0.24 | Hurt |
| replace gradient-based grouping with instance-number grouping (ablation) | GBG, ResNeXT-50 · ImageNet-LT | top-1 accuracy: 58.61±0.37 -> 57.44±0.38 | Hurt |
| add gradient-balanced grouping to BCL (ablation) | BCL, ResNet-32 · CIFAR10-LT IF=100 | accuracy: 84.07 -> 84.20 | Helped |
Surprising findings
- Applying MOO directly over per-class losses drops CIFAR10-LT accuracy to 74.58% (BCL+MOO) because tail classes are often absent from batches, so the objective set is not fixed.
- Using four groups performs best; assigning each class as its own objective significantly undermines performance.
Limitations
- Number of groups and grouping must be hand-tuned as hyperparameters.
- Gains over BCL on CIFAR10/100-LT and iNaturalist are small (e.g., 0.13% from GBG alone on BCL).
Experimental setup
Datasets: CIFAR10-LT CIFAR100-LT ImageNet-LT iNaturalist 2018
Benchmarks: CIFAR10-LT CIFAR100-LT ImageNet-LT iNaturalist 2018
Metrics: top-1 accuracy
Baselines: CE CB-Focal LDAM-DRW BBN SSD PaCo RISDA FDC DisVar BCL RIDE LADE
Builds on: MGDA BCL Balanced Contrastive Learning class-balanced loss spectral clustering
Closest papers by meaning
- Long-Tailed Recognition via Information-Preservable Two-Stage Learning (NeurIPS 2025 · similarity 0.89)
- Exploring Weight Balancing on Long-Tailed Recognition Problem (ICLR 2024 · similarity 0.89)
- Long-tailed Recognition with Model Rebalancing (NeurIPS 2025 · similarity 0.89)
- Decoupled Optimisation for Long-Tailed Visual Recognition (AAAI 2024 · similarity 0.88)
- Breaking Long-Tailed Learning Bottlenecks: A Controllable Paradigm with Hypernetwork-Generated Diverse Experts (NeurIPS 2024 · similarity 0.88)
- Difficulty-aware Balancing Margin Loss for Long-tailed Recognition (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.