TPAMI 2023 papers on Architectures and efficiency
45 of the 664 papers published in TPAMI 2023 (6.8%) are on Architectures and efficiency, compared with 6.3% at TPAMI 2022. They fall into 7 topics.
See TPAMI 2023 in the interactive map
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
Attention and transformers · 11 papers
- Multimodal Learning With Transformers: A Survey (1,280 citations)
- UniFormer: Unifying Convolution and Self-Attention for Visual Recognition (693 citations)
- GFNet: Global Filter Networks for Visual Recognition (579 citations)
- MetaFormer Baselines for Vision (418 citations)
- Dual Vision Transformer (199 citations)
- Dynamic Spatial Sparsification for Efficient Vision Transformers and Convolutional Neural Networks (80 citations)
- Vicinity Vision Transformer (66 citations)
- Dynamic Unary Convolution in Transformers (47 citations)
- PSLT: A Light-Weight Vision Transformer With Ladder Self-Attention and Progressive Shift (36 citations)
- What Makes for Hierarchical Vision Transformer? (21 citations)
- Linear Complexity Self-Attention with $3^{\text{rd}}$ Order Polynomials (11 citations)
RNNs and normalization · 11 papers
- Normalization Techniques in Training DNNs: Methodology, Analysis and Application (666 citations)
- Accurate and Efficient Stereo Matching via Attention Concatenation Volume (130 citations)
- A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond (118 citations)
- Sparse Bayesian Learning for End-to-End EEG Decoding (91 citations)
- Learning Interpretable Rules for Scalable Data Representation and Classification (58 citations)
- REDRESS: Generating Compressed Models for Edge Inference Using Tsetlin Machines (40 citations)
- BNET: Batch Normalization With Enhanced Linear Transformation (27 citations)
- PWLU: Learning Specialized Activation Functions With the Piecewise Linear Unit (18 citations)
- Geometric Back-propagation in Morphological Neural Networks (13 citations)
- Prioritized Subnet Sampling for Resource-Adaptive Supernet Training (5 citations)
- Towards a Deeper Understanding of Global Covariance Pooling in Deep Learning: An Optimization Perspective
Pruning and efficient tuning · 8 papers
- Structured Pruning for Deep Convolutional Neural Networks: A Survey (471 citations)
- Performance-Aware Approximation of Global Channel Pruning for Multitask CNNs (45 citations)
- Training Compact CNNs for Image Classification Using Dynamic-Coded Filter Fusion (45 citations)
- Compact Neural Network via Stacking Hybrid Units (15 citations)
- An Integrated Fast Hough Transform for Multidimensional Data (14 citations)
- Lottery Jackpots Exist in Pre-Trained Models (14 citations)
- Single-Path Bit Sharing for Automatic Loss-Aware Model Compression (14 citations)
- Earning Extra Performance From Restrictive Feedbacks (2 citations)
Neural architecture search · 6 papers
- MNGNAS: Distilling Adaptive Combination of Multiple Searched Networks for One-Shot Neural Architecture Search (84 citations)
- Understanding and Accelerating Neural Architecture Search With Training-Free and Theory-Grounded Metrics (32 citations)
- Differentiable Image Data Augmentation and Its Applications: A Survey (25 citations)
- Adaptive Search-and-Training for Robust and Efficient Network Pruning (20 citations)
- DIY Your EasyNAS for Vision: Convolution Operation Merging, Map Channel Reducing, and Search Space to Supernet Conversion Tooling (4 citations)
- DNA Family: Boosting Weight-Sharing NAS With Block-Wise Supervisions
Spiking networks · 5 papers
- Attention Spiking Neural Networks (304 citations)
- Fast-SNN: Fast Spiking Neural Network by Converting Quantized ANN (132 citations)
- A Hybrid Neural Coding Approach for Pattern Recognition With Spiking Neural Networks (59 citations)
- Event Transformer$^+$. A Multi-Purpose Solution for Efficient Event Data Processing (23 citations)
- CQ+ Training: Minimizing Accuracy Loss in Conversion From Convolutional Neural Networks to Spiking Neural Networks
