CVPR 2024 accepted papers on Architectures and efficiency

Attention and transformers · 42 papers

• RMT: Retentive Networks Meet Vision Transformers (229 citations)
• Adapt or Perish: Adaptive Sparse Transformer with Attentive Feature Refinement for Image Restoration (210 citations)
• SHViT: Single-Head Vision Transformer with Memory Efficient Macro Design (179 citations)
• Efficient LoFTR: Semi-Dense Local Feature Matching with Sparse-Like Speed (177 citations)
• MoMask: Generative Masked Modeling of 3D Human Motions (171 citations)
• ViT-CoMer: Vision Transformer with Convolutional Multi-scale Feature Interaction for Dense Predictions (129 citations)
• ParameterNet: Parameters Are All You Need for Large-scale Visual Pretraining of Mobile Networks (101 citations)
• Correlation-aware Coarse-to-fine MLPs for Deformable Medical Image Registration (62 citations)
• BoQ: A Place is Worth a Bag of Learnable Queries (42 citations)
• HomoFormer: Homogenized Transformer for Image Shadow Removal (39 citations)
• H-ViT: A Hierarchical Vision Transformer for Deformable Image Registration (37 citations)
• Sharingan: A Transformer Architecture for Multi-Person Gaze Following (31 citations)

and 30 more in the interactive map.

RNNs and normalization · 29 papers

• Rewrite the Stars (612 citations)
• Selective-Stereo: Adaptive Frequency Information Selection for Stereo Matching (103 citations)
• PARA-Drive: Parallelized Architecture for Real-time Autonomous Driving (61 citations)
• State Space Models for Event Cameras (44 citations)
• SchurVINS: Schur Complement-Based Lightweight Visual Inertial Navigation System (36 citations)
• Improved Implicit Neural Representation with Fourier Reparameterized Training (28 citations)
• MCNet: Rethinking the Core Ingredients for Accurate and Efficient Homography Estimation (27 citations)
• PointBeV: A Sparse Approach for BeV Predictions (23 citations)
• Batch Normalization Alleviates the Spectral Bias in Coordinate Networks (19 citations)
• Generalized Large-Scale Data Condensation via Various Backbone and Statistical Matching (19 citations)
• Memory-Scalable and Simplified Functional Map Learning (13 citations)
• PixelRNN: In-pixel Recurrent Neural Networks for End-to-end-optimized Perception with Neural Sensors (12 citations)

and 17 more in the interactive map.

Pruning and efficient tuning · 22 papers

• Zero-TPrune: Zero-Shot Token Pruning through Leveraging of the Attention Graph in Pre-Trained Transformers (39 citations)
• MTLoRA: Low-Rank Adaptation Approach for Efficient Multi-Task Learning (33 citations)
• Gradient-based Parameter Selection for Efficient Fine-Tuning (29 citations)
• BA-SAM: Scalable Bias-Mode Attention Mask for Segment Anything Model (20 citations)
• Low-Rank Rescaled Vision Transformer Fine-Tuning: A Residual Design Approach (20 citations)
• Spanning Training Progress: Temporal Dual-Depth Scoring (TDDS) for Enhanced Dataset Pruning (19 citations)
• BilevelPruning: Unified Dynamic and Static Channel Pruning for Convolutional Neural Networks (18 citations)
• LORS: Low-rank Residual Structure for Parameter-Efficient Network Stacking (16 citations)
• Time- Memory- and Parameter-Efficient Visual Adaptation (16 citations)
• UniPT: Universal Parallel Tuning for Transfer Learning with Efficient Parameter and Memory (16 citations)
• Auto-Train-Once: Controller Network Guided Automatic Network Pruning from Scratch (15 citations)
• Resource-Efficient Transformer Pruning for Finetuning of Large Models (14 citations)

and 10 more in the interactive map.

CNN design · 13 papers

• EMCAD: Efficient Multi-scale Convolutional Attention Decoding for Medical Image Segmentation (506 citations)
• InceptionNeXt: When Inception Meets ConvNeXt (401 citations)
• XFeat: Accelerated Features for Lightweight Image Matching (185 citations)
• PeLK: Parameter-efficient Large Kernel ConvNets with Peripheral Convolution (83 citations)
• Content-Adaptive Non-Local Convolution for Remote Sensing Pansharpening (51 citations)
• RepViT: Revisiting Mobile CNN From ViT Perspective (30 citations)
• LSK3DNet: Towards Effective and Efficient 3D Perception with Large Sparse Kernels (29 citations)
• Improving Depth Completion via Depth Feature Upsampling (27 citations)
• Comparing the Decision-Making Mechanisms by Transformers and CNNs via Explanation Methods (17 citations)
• Making Vision Transformers Truly Shift-Equivariant (9 citations)
• AdaShift: Learning Discriminative Self-Gated Neural Feature Activation With an Adaptive Shift Factor (2 citations)
• Affine Equivariant Networks Based on Differential Invariants (2 citations)

and 1 more in the interactive map.

Quantization · 7 papers

• Instance-Aware Group Quantization for Vision Transformers (20 citations)
• JointSQ: Joint Sparsification-Quantization for Distributed Learning (18 citations)
• BiPer: Binary Neural Networks using a Periodic Function (12 citations)
• AdaBM: On-the-Fly Adaptive Bit Mapping for Image Super-Resolution (11 citations)
• Retraining-Free Model Quantization via One-Shot Weight-Coupling Learning (7 citations)
• A&B BNN: Add&Bit-Operation-Only Hardware-Friendly Binary Neural Network (6 citations)
• PikeLPN: Mitigating Overlooked Inefficiencies of Low-Precision Neural Networks (6 citations)

Neural architecture search · 4 papers

• AZ-NAS: Assembling Zero-Cost Proxies for Network Architecture Search (27 citations)
• Boosting Order-Preserving and Transferability for Neural Architecture Search: a Joint Architecture Refined Search and Fine-tuning Approach (5 citations)
• Building Optimal Neural Architectures using Interpretable Knowledge (3 citations)
• Insights from the Use of Previously Unseen Neural Architecture Search Datasets

Spiking networks · 2 papers

• SpikingResformer: Bridging ResNet and Vision Transformer in Spiking Neural Networks (82 citations)
• Are Conventional SNNs Really Efficient? A Perspective from Network Quantization (10 citations)

Mixture of experts · 1 paper

• Multi-Task Dense Prediction via Mixture of Low-Rank Experts (50 citations)