Research map / Architectures and efficiency
Pruning and efficient tuning: research map
1,147 accepted papers on Pruning and efficient tuning in Architectures and efficiency, from ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026), grouped into 3 clusters and 11 approaches. The busiest year so far is 2026.
Within Architectures and efficiency, its share grew from 14.2% in 2023–24 to 15.7% in 2025–26 (278 → 556 papers at ICML, NeurIPS, CVPR and AAAI, the venues with data for all four years).
Explore Pruning and efficient tuning in the interactive map
Working on something in this topic? Describe your idea in scime atlas to see which approach it falls under, the closest papers by meaning and how crowded the spot has become.
Approaches and key papers
fine tuning · lora · adaptation · 655 papers
Approaches in this cluster:
- Low-rank compression and merging (226 papers)
Compresses and adapts models with low-rank factorization and weight merging. - Adapter-based parameter-efficient tuning (149 papers)
Adds lightweight adapters and reparameterized modules for memory- and parameter-efficient fine-tuning. - Parameter-efficient transfer analysis (97 papers)
Analyzes and unifies transfer methods, from theoretical studies to multi-task model fusion. - LoRA variants and theory (121 papers)
Improves low-rank adaptation through new parameterizations, optimization fixes, and expressiveness analysis. - Efficient ViT adaptation (62 papers)
Adapts pre-trained vision transformers with low-rank and factorized parameter-efficient tuning.
Most cited and most cited since 2024:
- Low-Shot Learning With Imprinted Weights (CVPR 2018 · 558 citations)
- Fine-Tuning Convolutional Neural Networks for Biomedical Image Analysis: Actively and Incrementally (CVPR 2017 · 429 citations)
- PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models (NeurIPS 2024 · 72 citations)
- VMT-Adapter: Parameter-Efficient Transfer Learning for Multi-Task Dense Scene Understanding (AAAI 2024 · 62 citations)
sparsity · compression · structured · 426 papers
Approaches in this cluster:
- Structured network pruning (162 papers)
Prunes filters and channels with budget-aware regularization and probabilistic or continuous criteria. - Sparse training from scratch (95 papers)
Trains sparse networks with dynamic or random sparsity instead of pruning dense models. - Joint pruning and quantization (78 papers)
Combines pruning, quantization, and low-rank decomposition for model compression. - Layer-wise pruning of LLMs (91 papers)
Allocates sparsity across layers for one-shot or structured pruning of large language models.
Most cited and most cited since 2024:
- Pruning Convolutional Neural Networks for Resource Efficient Inference (ICLR 2017 · 1,321 citations)
- Importance Estimation for Neural Network Pruning (CVPR 2019 · 956 citations)
- IRPruneDet: Efficient Infrared Small Target Detection via Wavelet Structure-Regularized Soft Channel Pruning (AAAI 2024 · 155 citations)
- Spanning Training Progress: Temporal Dual-Depth Scoring (TDDS) for Enhanced Dataset Pruning (CVPR 2024 · 19 citations)
lottery · tickets · winning · 66 papers
Approaches in this cluster:
- Lottery ticket discovery (42 papers)
Finds trainable sparse subnetworks via iterative magnitude pruning and analyzes winning tickets. - Pruning at initialization (24 papers)
Prunes before training using data-free scores and examines when sparse initializations suffice.
Most cited and most cited since 2024:
- The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks (ICLR 2019 · 1,303 citations)
- Rethinking the Value of Network Pruning (ICLR 2019 · 958 citations)
- Finding Lottery Tickets in Vision Models via Data-driven Spectral Foresight Pruning (CVPR 2024 · 9 citations)
- No Free Prune: Information-Theoretic Barriers to Pruning at Initialization (ICML 2024 · 2 citations)
Related topics in Architectures and efficiency
- Attention and transformers (2,296)
- Neural architecture search (380)
- Quantization (646)
- RNNs and normalization (2,827)
- Spiking networks (289)
- Mixture of experts (300)
- CNN design (853)
