Research map

Architectures and efficiency: research map

Transformers, MoE, pruning, quantization and architecture search.

8,738 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026), in 8 topics. Within all five venues, its share held steady from 7.4% in 2023–24 to 7.8% in 2025–26 (1,954 → 3,543 papers at ICML, NeurIPS, CVPR and AAAI, the venues with data for all four years).

Explore Architectures and efficiency in the interactive map

Topics

• RNNs and normalization · 2,827 papers
  Efficient network compression and training, Dataset distillation and data-efficient training, Neural scaling laws
• Attention and transformers · 2,296 papers
  Sparse and structured attention, Attention head and block redesign, Efficient image transformers
• Pruning and efficient tuning · 1,147 papers
  Low-rank compression and merging, Adapter-based parameter-efficient tuning, Parameter-efficient transfer analysis
• CNN design · 853 papers
  Filter design and trained-filter analysis, Compact CNN architectures, Theory of convolutional inductive bias
• Quantization · 646 papers
  Post-training weight quantization, Post-training quantization of LLMs, Post-training quantization of ViTs
• Neural architecture search · 380 papers
  Efficient NAS search strategies, Bi-level differentiable NAS, One-shot and few-shot supernets
• Mixture of experts · 300 papers
  Expert routing and specialization, Sparse MoE as regularization and attention, Efficient MoE language model systems
• Spiking networks · 289 papers
  Training deep spiking networks, Spiking transformers, Event-camera spiking networks

Most cited papers in Architectures and efficiency

• Densely Connected Convolutional Networks (CVPR 2017 · 45,904 citations)
• Rethinking the Inception Architecture for Computer Vision (CVPR 2016 · 31,549 citations)
• Squeeze-and-Excitation Networks (CVPR 2018 · 30,472 citations)
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (ICLR 2021 · 21,812 citations)
• PyTorch: An Imperative Style, High-Performance Deep Learning Library (NeurIPS 2019 · 15,937 citations)
• Masked Autoencoders Are Scalable Vision Learners (CVPR 2022 · 7,924 citations)
• Learning Transferable Architectures for Scalable Image Recognition (CVPR 2018 · 6,603 citations)
• Neural Architecture Search with Reinforcement Learning (ICLR 2017 · 4,370 citations)
• Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference (CVPR 2018 · 3,940 citations)
• Social LSTM: Human Trajectory Prediction in Crowded Spaces (CVPR 2016 · 3,627 citations)
• Convolutional Sequence to Sequence Learning (ICML 2017 · 2,604 citations)
• On Calibration of Modern Neural Networks (ICML 2017 · 2,519 citations)