Research map / Architectures and efficiency / Pruning and efficient tuning
Balancing Model Efficiency and Performance: Adaptive Pruner for Long-tailed Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LTAP, a multi-criteria, class-aware adaptive pruning method that reweights pruning criteria by per-class accuracy changes to protect tail-class parameters on long-tailed image classification.
Problem
Standard neural network pruning is blind to class imbalance and can remove parameters critical for tail classes in long-tailed datasets, degrading tail accuracy.
Approach
Combines five importance criteria (magnitude, avg magnitude, cosine similarity, first- and second-order Taylor) with a learnable class-criteria weight matrix updated by per-class validation accuracy changes (LT-Vote), applied in progressive multi-stage pruning that keeps 30% of parameters.
Key results
- On CIFAR-100-LT IR=50, BS + LTAP reaches 34.1% tail accuracy and 4.1 C/F versus 23.8% tail and 2.2 C/F for BS + RReg, with FLOPs reduced by 77.4% relative to BS.
- On ImageNet-LT, DBLP + LTAP reaches 93.8% of baseline accuracy with about 70% FLOPs reduction and C/F of 3.1.
- On CIFAR-100-LT IR=100, LDAM-DRW + LTAP attains 86.8% overall accuracy relative accuracy C of 86.8% vs 82.0% for LDAM-DRW + RReg, at similar FLOPs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply LTAP rather than RReg with BS | BS on CIFAR-100-LT · CIFAR-100-LT (IR=50) | Tail accuracy (%): 23.8 -> 34.1 | Helped |
| apply LTAP rather than RReg with BS | BS on CIFAR-100-LT · CIFAR-100-LT (IR=100) | Tail accuracy (%): 19.0 -> 23.2 | Helped |
| apply LTAP rather than RReg with BS | BS on ImageNet-LT · ImageNet-LT | Tail accuracy (%): 18.2 -> 30.1 | Helped |
| apply LTAP rather than RReg with BS | BS on iNaturalist 2018 · iNaturalist 2018 | Tail accuracy (%): 44.8 -> 60.7 | Helped |
| add tail-class protection (ablation) | CE + LTAP on CIFAR-100-LT · CIFAR-100-LT (IR=50) | Tail accuracy (%): 4.5 -> 7.2 | Helped |
| add tail-class protection (ablation) | LA + LTAP on CIFAR-100-LT · CIFAR-100-LT (IR=50) | Tail accuracy (%): 30.0 -> 38.4 | Helped |
| add tail-class protection (ablation) | CE + LTAP on CIFAR-100-LT · CIFAR-100-LT (IR=100) | Tail accuracy (%): 0.5 -> 2.5 | Helped |
| add tail-class protection (ablation) | LA + LTAP on CIFAR-100-LT · CIFAR-100-LT (IR=100) | Tail accuracy (%): 18.6 -> 22.6 | Helped |
Surprising findings
- Traditional pruning (ATO) causes catastrophic tail-class accuracy drops, e.g., CE + RReg reaches 0.83% tail accuracy at IR=50 with 52.1% FLOPs.
Limitations
- Theoretical results rely on loose assumptions (e.g., logarithmic performance-parameter relation) and asymptotic bounds, with proofs that are informal.
- Reported results are on image classification only, with ResNet backbones; no evaluation on other architectures or modalities.
- Compute and hardware details and variance over seeds are not reported.
Experimental setup
Datasets: CIFAR-100-LT ImageNet-LT iNaturalist 2018
Benchmarks: CIFAR-100-LT ImageNet-LT iNaturalist 2018
Metrics: head/medium/tail/overall accuracy FLOPs ratio accuracy-to-FLOPs ratio C/F
Baselines: ATO RReg Balanced Softmax LDAM-DRW DBLP Logit Adjustment
Builds on: Balanced Softmax LDAM-DRW DBLP ATO RReg
Closest papers by meaning
- Long-Tailed Learning as Multi-Objective Optimization (AAAI 2024 · similarity 0.87)
- Inverse Weight-Balancing for Deep Long-Tailed Learning (AAAI 2024 · similarity 0.86)
- Exploring Weight Balancing on Long-Tailed Recognition Problem (ICLR 2024 · similarity 0.85)
- LT-Soups: Bridging Head and Tail Classes via Subsampled Model Soups (NeurIPS 2025 · similarity 0.85)
- Breaking Long-Tailed Learning Bottlenecks: A Controllable Paradigm with Hypernetwork-Generated Diverse Experts (NeurIPS 2024 · similarity 0.85)
- Long-tailed Recognition with Model Rebalancing (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.