AAAI 2025 accepted papers on Architectures and efficiency

RNNs and normalization · 53 papers

• HEROS-GAN: Honed-Energy Regularized and Optimal Supervised GAN for Enhancing Accuracy and Range of Low-Cost Accelerometers (17 citations)
• PowerMLP: An Efficient Version of KAN (11 citations)
• Light-T2M: A Lightweight and Fast Model for Text-to-motion Generation (8 citations)
• Poplar: Efficient Scaling of Distributed DNN Training on Heterogeneous GPU Clusters (6 citations)
• Scalable Quantum-Inspired Optimization Through Dynamic Qubit Compression (6 citations)
• L3TC: Leveraging RWKV for Learned Lossless Low-Complexity Text Compression (4 citations)
• M3Net: Efficient Time-Frequency Integration Network with Mirror Attention for Audio Classification on Edge (4 citations)
• POPoS: Improving Efficient and Robust Facial Landmark Detection with Parallel Optimal Position Search (4 citations)
• DrugHash: Hashing Based Contrastive Learning for Virtual Screening (3 citations)
• END^2: Robust Dual-Decoder Watermarking Framework Against Non-Differentiable Distortions (3 citations)
• Feature Clipping for Uncertainty Calibration (3 citations)
• HomoMatcher: Achieving Dense Feature Matching with Semi-Dense Efficiency by Homography Estimation (3 citations)

and 41 more in the interactive map.

Attention and transformers · 42 papers

• Can We Get Rid of Handcrafted Feature Extractors? SparseViT: Nonsemantics-Centered, Parameter-Efficient Image Manipulation Localization Through Spare-Coding Transformer (40 citations)
• Rethinking Transformer-Based Blind-Spot Network for Self-Supervised Image Denoising (32 citations)
• Occlusion-Embedded Hybrid Transformer for Light Field Super-Resolution (27 citations)
• Robust Tracking via Mamba-based Context-aware Token Learning (27 citations)
• Cross-modulated Attention Transformer for RGBT Tracking (26 citations)
• SIGMA: Selective Gated Mamba for Sequential Recommendation (25 citations)
• MIETT: Multi-Instance Encrypted Traffic Transformer for Encrypted Traffic Classification (20 citations)
• TAMER: Tree-Aware Transformer for Handwritten Mathematical Expression Recognition (16 citations)
• Efficient Attention-Sharing Information Distillation Transformer for Lightweight Single Image Super-Resolution (14 citations)
• SparX: A Sparse Cross-Layer Connection Mechanism for Hierarchical Vision Mamba and Transformer Networks (13 citations)
• Detail Matters: Mamba-Inspired Joint Unfolding Network for Snapshot Spectral Compressive Imaging (12 citations)
• Motion-adaptive Transformer for Event-based Image Deblurring (12 citations)

and 30 more in the interactive map.

Pruning and efficient tuning · 23 papers

• TinySAM: Pushing the Envelope for Efficient Segment Anything Model (39 citations)
• Numerical Pruning for Efficient Autoregressive Models (11 citations)
• Pruning Large Language Models with Semi-Structural Adaptive Sparse Training (10 citations)
• CALLIC: Content Adaptive Learning for Lossless Image Compression (8 citations)
• From PEFT to DEFT: Parameter Efficient Finetuning for Reducing Activation Density in Transformers (7 citations)
• PAT: Pruning-Aware Tuning for Large Language Models (5 citations)
• No More Tuning: Prioritized Multi-Task Learning with Lagrangian Differential Multiplier Methods (4 citations)
• Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective (4 citations)
• Enhancing Low-Rank Adaptation with Recoverability-Based Reinforcement Pruning for Object Counting (3 citations)
• Practical Offloading for Fine-Tuning LLM on Commodity GPU via Learned Sparse Projectors (3 citations)
• Genomics Data Lossless Compression with (S, K)-Mer Encoding and Deep Neural Networks (2 citations)
• Singular Value Scaling: Efficient Generative Model Compression via Pruned Weights Refinement (2 citations)

and 11 more in the interactive map.

Quantization · 15 papers

• ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models (16 citations)
• GradQ-ViT: Robust and Efficient Gradient Quantization for Vision Transformers (16 citations)
• JAQ: Joint Efficient Architecture Design and Low-Bit Quantization with Hardware-Software Co-Exploration (6 citations)
• VQ4DiT: Efficient Post-Training Vector Quantization for Diffusion Transformers (6 citations)
• Adaptive Dataset Quantization (5 citations)
• AIQViT: Architecture-Informed Post-Training Quantization for Vision Transformers (5 citations)
• MimiQ: Low-Bit Data-Free Quantization of Vision Transformers with Encouraging Inter-Head Attention Similarity (4 citations)
• MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Models (4 citations)
• QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overhead (4 citations)
• Pushing the Limits of BFP on Narrow Precision LLM Inference (3 citations)
• OAC: Output-adaptive Calibration for Accurate Post-training Quantization (2 citations)
• Thinking in Granularity: Dynamic Quantization for Image Super-Resolution by Intriguing Multi-Granularity Clues (2 citations)

and 3 more in the interactive map.

CNN design · 10 papers

• EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba (165 citations)
• Efficient Indoor Depth Completion Network Using Mask-adaptive Gated Convolution (3 citations)
• FreeNet: Liberating Depth-Wise Separable Operations for Building Faster Mobile Vision Architectures (3 citations)
• EDENet: Echo Direction Encoding Network for Place Recognition Based on Ground Penetrating Radar (2 citations)
• ParseCaps: An Interpretable Parsing Capsule Network for Medical Image Diagnosis (2 citations)
• LayerAct: Advanced Activation Mechanism for Robust Inference of CNNs (1 citation)
• The Master Key Filters Hypothesis: Deep Filters Are General (1 citation)
• Alignment of CNN and Human Judgments of Geometric and Topological Concepts
• Beyond Skip Connection: Pooling and Unpooling Design for Elimination Singularities
• Don’t Think It Twice: Exploit Shift Invariance for Efficient Online Streaming Inference of CNNs

Spiking networks · 8 papers

• FSTA-SNN:Frequency-Based Spatial-Temporal Attention Module for Spiking Neural Networks (14 citations)
• Spike2Former: Efficient Spiking Transformer for High-performance Image Segmentation (13 citations)
• SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Models (10 citations)
• Adaptive Calibration: A Unified Conversion Framework of Spiking Neural Networks (6 citations)
• Advancing Spiking Neural Networks Towards Multiscale Spatiotemporal Interaction Learning (6 citations)
• Leveraging Asynchronous Spiking Neural Networks for Ultra Efficient Event-Based Visual Processing (3 citations)
• Self-Attentive Spatio-Temporal Calibration for Precise Intermediate Layer Matching in ANN-to-SNN Distillation (2 citations)
• Towards Accurate Binary Spiking Neural Networks: Learning with Adaptive Gradient Modulation Mechanism

Neural architecture search · 7 papers

• Design Principle Transfer in Neural Architecture Search via Large Language Models (13 citations)
• ParZC: Parametric Zero-Cost Proxies for Efficient NAS (9 citations)
• Efficient Few-Shot Neural Architecture Search by Counting the Number of Nonlinear Functions (3 citations)
• Resource Constrained Pathfinding with Enhanced Bidirectional A* Search (2 citations)
• HEP-NAS: Towards Efficient Few-shot Neural Architecture Search via Hierarchical Edge Partitioning (1 citation)
• AtomNet: Designing Tiny Models from Operators Under Extreme MCU Constraints
• Efficient Image Similarity Search with Quadtrees (Student Abstract)

Mixture of experts · 4 papers

• Rethinking U-Net: Task-Adaptive Mixture of Skip Connections for Enhanced Medical Image Segmentation (8 citations)
• EWMoE: An Effective Model for Global Weather Forecasting with Mixture-of-Experts (6 citations)
• BigMac: A Communication-Efficient Mixture-of-Experts Model Structure for Fast Training and Inference (5 citations)
• Bagging-Expert Network for Multi-Task Learning: A Depolarization Solution in Multi-Gate Mixture-of-Experts (3 citations)