AAAI 2022 accepted papers on Architectures and efficiency
87 of the 1,319 papers accepted at AAAI 2022 (6.6%) are on Architectures and efficiency, compared with 7.7% at AAAI 2021. They fall into 8 topics.
See AAAI 2022 in the interactive map
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
Attention and transformers · 30 papers
- TransFG: A Transformer Architecture for Fine-Grained Recognition (523 citations)
- Vision Transformers Are Robust Learners (213 citations)
- Evo-ViT: Slow-Fast Token Evolution for Dynamic Vision Transformer (180 citations)
- Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual Understanding (155 citations)
- Frozen Pretrained Transformers as Universal Computation Engines (111 citations)
- Width & Depth Pruning for Vision Transformers (94 citations)
- Sparse MLP for Image Recognition: Is Self-Attention Really Necessary? (87 citations)
- When Shift Operation Meets Vision Transformer: An Extremely Simple Alternative to Attention Mechanism (84 citations)
- Less Is More: Pay Less Attention in Vision Transformers (81 citations)
- Handwritten Mathematical Expression Recognition via Attention Aggregation Based Bi-directional Mutual Learning (73 citations)
- Pale Transformer: A General Vision Transformer Backbone with Pale-Shaped Attention (62 citations)
- Towards End-to-End Image Compression and Analysis with Transformers (60 citations)
and 18 more in the interactive map.
RNNs and normalization · 24 papers
- Towards Light-Weight and Real-Time Line Segment Detection (91 citations)
- Up to 100x Faster Data-Free Knowledge Distillation (79 citations)
- JFB: Jacobian-Free Backpropagation for Implicit Networks (52 citations)
- Neuro-Symbolic Inductive Logic Programming with Logical Neural Networks (42 citations)
- AlphaHoldem: High-Performance Artificial Intelligence for Heads-Up No-Limit Poker via End-to-End Reinforcement Learning (41 citations)
- SFSRNet: Super-resolution for Single-Channel Audio Source Separation (29 citations)
- EtinyNet: Extremely Tiny Network for TinyML (27 citations)
- Non-autoregressive Translation with Layer-Wise Prediction and Deep Supervision (25 citations)
- Post-OCR Document Correction with Large Ensembles of Character Sequence-to-Sequence Models (18 citations)
- Reverse Differentiation via Predictive Coding (17 citations)
- ErfAct and Pserf: Non-monotonic Smooth Trainable Activation Functions (13 citations)
- DeepHardMark: Towards Watermarking Neural Network Hardware (12 citations)
and 12 more in the interactive map.
CNN design · 15 papers
- Topology-Aware Convolutional Neural Network for Efficient Skeleton-Based Action Recognition (177 citations)
- Pan-Sharpening with Customized Transformer and Invertible Neural Network (128 citations)
- Dynamic Spatial Propagation Network for Depth Completion (126 citations)
- LAGConv: Local-Context Adaptive Convolution Kernels with Global Harmonic Bias for Pansharpening (118 citations)
- Feature Distillation Interaction Weighting Network for Lightweight Image Super-resolution (97 citations)
- DANets: Deep Abstract Networks for Tabular Data Classification and Regression (59 citations)
- Scaled ReLU Matters for Training Vision Transformers (38 citations)
- Two-Stage Octave Residual Network for End-to-End Image Compression (37 citations)
- Convolutional Neural Network Compression through Generalized Kronecker Product Decomposition (21 citations)
- Efficient Compact Bilinear Pooling via Kronecker Product (14 citations)
- Zero Stability Well Predicts Performance of Convolutional Neural Networks (14 citations)
- DeformRS: Certifying Input Deformations with Randomized Smoothing (9 citations)
and 3 more in the interactive map.
Pruning and efficient tuning · 9 papers
- Prior Gradient Mask Guided Pruning-Aware Fine-Tuning (38 citations)
- Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win (37 citations)
- BATUDE: Budget-Aware Neural Network Compression Based on Tucker Decomposition (23 citations)
- NoiseGrad — Enhancing Explanations by Introducing Stochasticity to Model Weights (23 citations)
- From Dense to Sparse: Contrastive Pruning for Better Pre-trained Language Model Compression (19 citations)
- Prune and Tune Ensembles: Low-Cost Ensemble Learning with Sparse Independent Subnetworks (19 citations)
- Adversarial Data Augmentation for Task-Specific Knowledge Distillation of Pre-trained Transformers (11 citations)
- Deep Fusing Pre-trained Models into Neural Machine Translation (7 citations)
- Towards Fully Sparse Training: Information Restoration with Spatial Similarity (4 citations)
Spiking networks · 4 papers
- Optimized Potential Initialization for Low-Latency Spiking Neural Networks (91 citations)
- SpikeConverter: An Efficient Conversion Framework Zipping the Gap between Artificial Neural Networks and Spiking Neural Networks (49 citations)
- Spiking Neural Networks with Improved Inherent Recurrence Dynamics for Sequential Learning (42 citations)
- PrivateSNN: Privacy-Preserving Spiking Neural Networks (26 citations)
Neural architecture search · 3 papers
- BM-NAS: Bilevel Multimodal Neural Architecture Search (31 citations)
- AutoBERT-Zero: Evolving BERT Backbone from Scratch (20 citations)
- Learning from Mistakes – a Framework for Neural Architecture Search (6 citations)
Quantization · 1 paper
Mixture of experts · 1 paper
- Go Wider Instead of Deeper (55 citations)
