Research map / Architectures and efficiency
Pruning and efficient tuning: research map
1,335 papers on Pruning and efficient tuning in Architectures and efficiency, from ICML, NeurIPS, ICLR, CVPR, AAAI, ICCV and ECCV, and the journals TPAMI, IJCV and JMLR (2016–2026), grouped into 3 clusters and 11 approaches. The busiest year so far is 2026.
What it is
Making networks smaller and cheaper: removing unneeded weights (pruning), training sparse networks, and adapting large models with few trainable parameters (adapters, LoRA). The lottery ticket hypothesis asks whether small trainable subnetworks exist from the start.
Terms you will see on this page:
- Pruning
- removing weights, channels or layers with little loss in accuracy.
- Lottery ticket hypothesis
- the idea that a dense network contains a small subnetwork that trains as well on its own.
- Adapter
- a small module inserted into a frozen model and trained for a new task.
Start reading with the most cited papers here:
- Pruning Convolutional Neural Networks for Resource Efficient Inference (ICLR 2017 · 1,321 citations)
- The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks (ICLR 2019 · 1,303 citations)
Within Architectures and efficiency, its share grew from 14.2% in 2023–24 to 15.7% in 2025–26 (278 → 556 papers at ICML, NeurIPS, ICLR, CVPR and AAAI, the venues with data for all four years).
Explore Pruning and efficient tuning in the interactive map
Working on something in this topic? Describe your idea in scime atlas to see which approach it falls under, the closest papers by meaning and how crowded the spot has become.
What changed recently
Approaches whose share of the topic moved most between 2023–24 and 2025–26 (ICML, NeurIPS, ICLR, CVPR and AAAI):
- Low-rank compression and merging rising: 13% → 27% of the topic (37 → 152 papers)
- LoRA variants and theory rising: 8% → 18% of the topic (21 → 99 papers)
- Layer-wise pruning of LLMs rising: 7% → 13% of the topic (19 → 70 papers)
- Structured network pruning falling: 15% → 5% of the topic (43 → 29 papers)
- Efficient ViT adaptation falling: 13% → 3% of the topic (35 → 16 papers)
Papers from 2025–2026 that are already cited most:
- TinySAM: Pushing the Envelope for Efficient Segment Anything Model (AAAI 2025 · 39 citations)
- Sign-In to the Lottery: Reparameterizing Sparse Training (NeurIPS 2025 · 0 citations)
- Sparse Training from Random Initialization: Aligning Lottery Ticket Masks using Weight Symmetry (ICML 2025 · 0 citations)
Approaches and key papers
Parameter-efficient fine-tuning · 774 papers
Keywords: fine tuning, lora, adaptation
Approaches in this cluster:
- Low-rank compression and merging (250 papers)
Compresses and adapts models with low-rank factorization and weight merging. - Adapter-based parameter-efficient tuning (169 papers)
Adds lightweight adapters and reparameterized modules for memory- and parameter-efficient fine-tuning. - Parameter-efficient transfer analysis (115 papers)
Analyzes and unifies transfer methods, from theoretical studies to multi-task model fusion. - LoRA variants and theory (136 papers)
Improves low-rank adaptation through new parameterizations, optimization fixes, and expressiveness analysis. - Efficient ViT adaptation (104 papers)
Adapts pre-trained vision transformers with low-rank and factorized parameter-efficient tuning.
Most cited and most cited since 2024:
- Low-Shot Learning With Imprinted Weights (CVPR 2018 · 558 citations)
- Fine-Tuning Convolutional Neural Networks for Biomedical Image Analysis: Actively and Incrementally (CVPR 2017 · 429 citations)
- PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models (NeurIPS 2024 · 72 citations)
- VMT-Adapter: Parameter-Efficient Transfer Learning for Multi-Task Dense Scene Understanding (AAAI 2024 · 62 citations)
Pruning and sparsity · 491 papers
Keywords: sparsity, compression, structured
Approaches in this cluster:
- Structured network pruning (201 papers)
Prunes filters and channels with budget-aware regularization and probabilistic or continuous criteria. - Sparse training from scratch (109 papers)
Trains sparse networks with dynamic or random sparsity instead of pruning dense models. - Joint pruning and quantization (90 papers)
Combines pruning, quantization, and low-rank decomposition for model compression. - Layer-wise pruning of LLMs (91 papers)
Allocates sparsity across layers for one-shot or structured pruning of large language models.
Most cited and most cited since 2024:
- Pruning Convolutional Neural Networks for Resource Efficient Inference (ICLR 2017 · 1,321 citations)
- Importance Estimation for Neural Network Pruning (CVPR 2019 · 956 citations)
- IRPruneDet: Efficient Infrared Small Target Detection via Wavelet Structure-Regularized Soft Channel Pruning (AAAI 2024 · 155 citations)
- Spanning Training Progress: Temporal Dual-Depth Scoring (TDDS) for Enhanced Dataset Pruning (CVPR 2024 · 19 citations)
Lottery tickets · 70 papers
Keywords: lottery, tickets, winning
Approaches in this cluster:
- Lottery ticket discovery (45 papers)
Finds trainable sparse subnetworks via iterative magnitude pruning and analyzes winning tickets. - Pruning at initialization (25 papers)
Prunes before training using data-free scores and examines when sparse initializations suffice.
Most cited and most cited since 2024:
- The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks (ICLR 2019 · 1,303 citations)
- Rethinking the Value of Network Pruning (ICLR 2019 · 958 citations)
- Finding Lottery Tickets in Vision Models via Data-driven Spectral Foresight Pruning (CVPR 2024 · 9 citations)
- No Free Prune: Information-Theoretic Barriers to Pruning at Initialization (ICML 2024 · 2 citations)
Related topics in Architectures and efficiency
- Attention and transformers (2,598)
- Neural architecture search (548)
- Quantization (785)
- RNNs and normalization (3,071)
- Spiking networks (333)
- Mixture of experts (318)
- CNN design (1,051)
