ICML 2026 accepted papers on Architectures and efficiency
679 of the 6,341 papers accepted at ICML 2026 (10.7%) are on Architectures and efficiency, compared with 9.7% at ICML 2025. They fall into 8 topics.
See ICML 2026 in the interactive map
Working on something in this area? Describe your idea in scime atlas to see the closest accepted papers by meaning and how crowded the spot has become.
Topics and papers
RNNs and normalization · 217 papers
- BioToken and BioFM – Biologically-Informed Tokenization Enables Accurate and Efficient Genomic Foundation Models (3 citations)
- ePC: Fast and Deep Predictive Coding in Digital Simulation (2 citations)
- TabICLv2: A Better, Faster, Scalable, and Open Tabular Foundation Model (2 citations)
- OneSearch: A Preliminary Exploration of the Unified End-to-End Generative Framework for E-commerce Search (1 citation)
- Protein Circuit Tracing via Cross-layer Transcoders (1 citation)
- scDataset: Scalable Data Loading for Deep Learning on Large-Scale Single-Cell Omics (1 citation)
- A Cartesian-3j Framework for Machine Learning Interatomic Potentials
- A Constrained Optimization Perspective of Unrolled Transformers
- A unified theory of feature learning in RNNs and DNNs
- Abstraction Induces the Brain Alignment of Language and Speech Models
- Active Budget Allocation for Efficient Scaling Law Estimation via Surrogate-Guided Pruning
- AdaGC: Enhancing LLM Pretraining Stability via Adaptive Gradient Clipping
and 205 more in the interactive map.
Attention and transformers · 215 papers
- (Sparse) Attention to the Details: Preserving Spectral Fidelity in ML-based Weather Forecasting Models
- A Capacity-Based Rationale for Multi-Head Attention
- A Provable Expressiveness Hierarchy in Hybrid Linear-Full Attention
- A Unified Sparse Attention via Multi-Granularity Compression
- Adaptive Physics Transformer with Fused Global-Local Attention for Subsurface Energy Systems
- Adaptive Visual Autoregressive Acceleration via Dual-Linkage Entropy Analysis
- AdaRoPE: Not All Attention Heads Should Rotate and Scale Equally
- AdaSplash-2: Faster Differentiable Sparse Attention
- Affine-Scaled Attention: Towards Flexible and Stable Transformer Attention
- Approximation Theory for Lipschitz Continuous Transformers
- Artificial Hippocampus Networks for Efficient Long-Context Modeling
- ATLAS: Learning to Optimally Memorize the Context at Test Time
and 203 more in the interactive map.
Pruning and efficient tuning · 77 papers
- A Theory of How Pretraining Shapes Inductive Bias in Fine-Tuning
- A3: an Analytical Low-Rank Approximation Framework for Attention
- Advancing SVD-based LLM Compression via Layer-Wise Error Model Search
- AGZO: Activation-Guided Zeroth-Order Optimization for LLM Fine-Tuning
- Balanced LoRA: Removing Parameter Invariance to Accelerate Convergence
- Breaking the Scale Barrier: One-Shot Knowledge Transfer via Frequency Transform
- Compress then Merge: From Multiple LoRAs into One Low-Rank Adapter
- CorrSteer: Generation-Time LLM Steering via Correlated Sparse Autoencoder Features
- CurvZO: Adaptive Curvature-Guided Sparse Zeroth-Order Optimization for Efficient LLM Fine-Tuning
- Decentralized Instruction Tuning: Conflict-Aware Splitting and Weight Merging
- Demystifying Mergeability: Interpretable Properties to Predict Model Merging Success
- Demystifying When Pruning Works via Representation Hierarchies
and 65 more in the interactive map.
Quantization · 65 papers
- 1-Bit Wonder: Improving QAT Performance in the Low-Bit Regime through K-Means Quantization
- AGoQ: Activation and Gradient Quantization for Memory-Efficient Distributed Training of LLMs
- ASTRA: Communication-Efficient Acceleration for Multi-Device Transformer Inference
- Attn-QAT: 4-Bit Attention With Quantization-Aware Training
- AutoQRA: Joint Optimization of Mixed-Precision Quantization and Low-rank Adapters for Efficient LLM Fine-Tuning
- BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Models
- CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMs
- Compute Where it Counts: Self Optimizing Language Models
- CrossQ: Task-Aligned Cross-Token Conditional Quantization for Late Interaction Retrieval
- Dissecting Quantization Error: A Concentration-Alignment Perspective
- ECO: Quantized Training without Full-Precision Master Weights
- FAIR-Calib: Frontier-Aware Instability-Reweighted Calibration for Post-Training Quantization of Diffusion Large Language Models
and 53 more in the interactive map.
Mixture of experts · 55 papers
- $L^3$: Large Lookup Layers
- Attention Sink Forges Native MoE in Attention Layers: Sink-Aware Training to Address Head Collapse
- Attribution-Guided and Coverage-Maximized Pruning for Structural MoE Compression
- Beyond Sunk Costs: Boosting LLM Pre-training Efficiency via Orthogonal Growth of Mixture-of-Experts
- Capacity-Aware Mixture Law Enables Efficient LLM Data Optimization
- DAG-MoE: From Simple Mixture to Structural Aggregation in Mixture-of-Experts
- DOT-MoE: Differentiable Optimal Transport for MoEfication
- DTop-p MoE: Sparsity-Controlled Dynamic Top-p MoE for Foundation Model Pre-training
- EasyBalance: Cross-Layer Load Balancing in Distributed MoE Inference
- Effective MoE-based LLM Compression by Exploiting Heterogeneous Inter-Group Experts Routing Frequency and Information Density
- Eigenvectors of Experts are Training-free Non-collapsing Routers
- ExpertWeaver: Unlocking the Inherent MoE in Dense LLMs with GLU Activation Patterns
and 43 more in the interactive map.
Spiking networks · 28 papers
- Bullet Trains: Parallelizing Training of Temporally Precise Spiking Neural Networks
- Efficiently Training Time-to-First-Spike Spiking Neural Networks from Scratch
- Event2Vec: Processing neuromorphic events directly by representations in vector space
- EvReflection: Event-Driven Micro-Dynamics for Reflection Removal
- SMM Transformer: Leveraging Spiking Neural Networks for Multimodal Tasks
- Spike-HTR: Spiking Neural Transformer for Handwritten Text Recognition
- TEFormer: Structured Bidirectional Temporal Enhancement Modeling in Spiking Transformers
- A$^2$SG: Adaptive and Asymmetric Surrogate Gradients for Training Deep Spiking Neural Networks
- AdaS: Adaptive Gradient Descent for Spiking Transformers
- Efficient Transformer Attention for SNNs via Hadamard Simplification
- Frequency Matching in Spiking Neural Networks for mmWave Sensing
- High-Fidelity ANN-to-SNN Conversion via Closed-Loop CKA Distillation
and 16 more in the interactive map.
CNN design · 18 papers
- A Hypertoroidal Covering for Perfect Color Equivariance
- A Theory of Contrastive Learning with Natural Images
- Activation-Free Backbones for Image Recognition: Polynomial Alternatives within MetaFormer-Style Vision Models
- An analytic theory of convolutional neural network inverse problems solvers
- Approximate Equivariance via Projection-Based Regularisation
- Connecting Independently Trained Modes via Layer-Wise Connectivity
- Flowers: A Warp Drive for Neural PDE Solvers
- FOVI: A biologically-inspired foveated interface for deep vision models
- Normalization Equivariance for Arbitrary Backbones, with Application to Image Denoising
- Platonic Transformers: A Solid Choice For Equivariance
- REViT: Roto-reflection Equivariant Convolutional Vision Transformer
- Riemannian Networks over Full-Rank Correlation Matrices
and 6 more in the interactive map.