Research map / Label-efficient and robust learning / Recognition and long-tail
Breaking Long-Tailed Learning Bottlenecks: A Controllable Paradigm with Hypernetwork-Generated Diverse Experts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PRL, a hypernetwork that generates diverse classifier-head experts over Dirichlet-sampled preference vectors, enabling test-time user-controlled head-tail trade-offs and robustness to test-time class-distribution shift on long-tailed benchmarks.
Problem
Long-tailed recognition methods assume fixed train/test class distributions and yield a fixed head-tail trade-off, failing under test distribution shift and unable to accommodate user preferences for head versus tail class performance.
Approach
A hypernetwork maps sampled preference vectors (Dirichlet noise) to classifier heads of several experts sharing a feature extractor, trained with a stochastic convex ensemble (log-sum-exp relaxation of worst-case loss) over cross-entropy, balanced softmax, and inverse softmax losses; at test time a user preference is passed through the hypernetwork to produce a dedicated model.
Key results
- On CIFAR100-LT, PRL reaches 65.6% top-1 at IR=10 versus 65.0% for LSC and 64.8% for BalPoE; on ImageNet-LT it reaches 60.8% versus 60.2% for LSC.
- Under unknown test distributions on ImageNet-LT, PRL achieves 57.5% in the backward-50 setting versus 57.0% for LSC.
- On CIFAR100-LT, changing the preference vector shifts results: R=(1.0,2.7) gives best many-shot accuracy while R=(1.9,1.1) favors few-shot classes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PRL instead of the strongest listed baseline | CIFAR100-LT IR=10 · CIFAR100-LT IR=10 | top-1 accuracy: 65.0 -> 65.6 | Helped |
| Use PRL instead of the strongest listed baseline | CIFAR100-LT IR=50 · CIFAR100-LT IR=50 | top-1 accuracy: 56.5 -> 57.3 | Helped |
| Use PRL instead of the strongest listed baseline | CIFAR100-LT IR=100 · CIFAR100-LT IR=100 | top-1 accuracy: 52.0 -> 52.8 | Helped |
| Use PRL instead of the strongest listed baseline | Places-LT · Places-LT | top-1 accuracy: 41.3 -> 41.6 | Helped |
| Use PRL instead of the strongest listed baseline | iNaturalist 2018 · iNaturalist 2018 | top-1 accuracy: 75.0 -> 75.1 | Helped |
| Use PRL instead of the strongest listed baseline | ImageNet-LT · ImageNet-LT | top-1 accuracy: 60.2 -> 60.8 | Helped |
| Use PRL instead of the strongest listed baseline | PRL · CIFAR100-LT IR=100 Forward-LT 50 | top-1 accuracy: 69.0 -> 69.5 | Helped |
| Use PRL instead of the strongest listed baseline | PRL · CIFAR100-LT IR=100 Uni. | top-1 accuracy: 51.6 -> 52.2 | Helped |
| Use PRL instead of the strongest listed baseline | PRL · CIFAR100-LT IR=100 Backward-LT 50 | top-1 accuracy: 48.0 -> 48.5 | Helped |
| Use PRL instead of the strongest listed baseline | PRL · ImageNet-LT Forward-LT 50 | top-1 accuracy: 72.2 -> 72.7 | Helped |
| Use PRL instead of the strongest listed baseline | PRL · ImageNet-LT Uni. | top-1 accuracy: 60.2 -> 60.7 | Helped |
| Use PRL instead of the strongest listed baseline | PRL · ImageNet-LT Backward-LT 50 | top-1 accuracy: 57.0 -> 57.5 | Helped |
Limitations
- Hypernetwork substantially increases trainable parameter count and may affect training stability.
- Gains over strongest baselines (LSC, BalPoE) are small, often under 1 percentage point.
- Test-time preference control is demonstrated only on CIFAR100-LT in detail; Places-LT and iNaturalist results are limited.
Experimental setup
Datasets: CIFAR100-LT ImageNet-LT iNaturalist 2018 Places-LT
Benchmarks: CIFAR100-LT ImageNet-LT iNaturalist 2018 Places-LT
Metrics: top-1 accuracy many-shot accuracy medium-shot accuracy few-shot accuracy
Baselines: MiSLAS Balanced Softmax LADE RIDE SADE Causal LSC BalPoE
Builds on: SADE RIDE Balanced Softmax hypernetworks Chebyshev scalarization
Closest papers by meaning
- Long-Tailed Learning as Multi-Objective Optimization (AAAI 2024 · similarity 0.88)
- Towards Heterogeneous Long-tailed Learning: Benchmarking, Metrics, and Toolbox (NeurIPS 2024 · similarity 0.88)
- A Square Peg in a Square Hole: Meta-Expert for Long-Tailed Semi-Supervised Learning (ICML 2025 · similarity 0.87)
- Two Fists, One Heart: Multi-Objective Optimization Based Strategy Fusion for Long-tailed Learning (ICML 2024 · similarity 0.87)
- LT-Soups: Bridging Head and Tail Classes via Subsampled Model Soups (NeurIPS 2025 · similarity 0.87)
- Distilling Long-tailed Datasets (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.