Research map / Label-efficient and robust learning / Recognition and long-tail

LT-Soups: Bridging Head and Tail Classes via Subsampled Model Soups

NeurIPS 2025 · Label-efficient and robust learning › Recognition and long-tail · approach: Classifier rebalancing and margin losses · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LT-Soups, a two-stage CLIP fine-tuning method that averages full fine-tuned models trained on progressively imbalanced subsets (with bootstrapping and EMA) and then retrains only the classifier with logit adjustment, to balance head and tail accuracy across varied long-tailed regimes.

Problem

PEFT methods such as LoRA and AdaptFormer preserve tail-class accuracy on CLIP for long-tailed data but degrade head-class accuracy, and traditional model soups underperform in tail-heavy settings; performance depends on both imbalance ratio and head-tail ratio.

Approach

Average weights of models fully fine-tuned on subsets with exponentially increasing imbalance ratios (recursive interpolation, bootstrapped replicates, EMA), then retrain only the classifier on the full data with logit adjustment, and introduce the head-tail ratio as a second axis for characterizing imbalance.

Key results

• On CIFAR100-LT (rho=100, eta=0.54), LT-Soups reaches 83.5 overall vs 81.3 for PEFT and 82.1 for Model Soups.
• On NIH-CXR-LT, LT-Soups achieves 39.3 overall and 30.9 few-shot vs 38.5 and 25.5 for PEFT (LIFT).
• On iNaturalist 2018, PEFT (LIFT) scores higher overall (79.1) than LT-Soups (78.2), so LT-Soups is not best on every dataset.

Measured effects reported in the paper

Surprising findings

• Classifier re-training has little to no effect on PEFT and Model Soups but helps LT-Soups.
• PEFT (LoRA) can replace full fine-tuning inside LT-Soups with matching accuracy (77.1 vs 77.2 on TinyImageNet-LT).

Limitations

• Head-tail split uses a fixed sample threshold, which may oversimplify class distribution structure.
• All experiments use a single random seed, so no error bars are reported.
• Higher computational cost than single-model fine-tuning (NM+1 training runs), and on some datasets (ImageNet-LT) total wall-clock time is longer than Full-FT.

Experimental setup

Datasets: CIFAR-100-LT TinyImageNet-LT Places-LT ImageNet-LT iNaturalist 2018 NIH-CXR-LT

Benchmarks: CIFAR100-LT Places-LT ImageNet-LT iNaturalist 2018 NIH-CXR-LT TinyImageNet-LT

Metrics: balanced accuracy many-shot accuracy medium-shot accuracy few-shot accuracy ECE Brier score NLL

Baselines: Full-FT PEFT (LIFT) Model Soups Linear Probing cRT BALLAD Decoder LPT

Builds on: model soups LIFT logit adjustment CLIP BALLAD

Closest papers by meaning

• SSE-SAM: Balancing Head and Tail Classes Gradually Through Stage-Wise SAM (AAAI 2025 · similarity 0.89)
• CORAL: Disentangling Latent Representations in Long-Tailed Diffusion (NeurIPS 2025 · similarity 0.87)
• Rethinking the Bias of Foundation Model under Long-tailed Distribution (ICML 2025 · similarity 0.87)
• Breaking Long-Tailed Learning Bottlenecks: A Controllable Paradigm with Hypernetwork-Generated Diverse Experts (NeurIPS 2024 · similarity 0.87)
• Learning from Neighbors: Category Extrapolation for Long-Tail Learning (CVPR 2025 · similarity 0.86)
• Long-Tailed Learning as Multi-Objective Optimization (AAAI 2024 · similarity 0.86)