AAAI 2021 accepted papers on Architectures and efficiency
128 of the 1,654 papers accepted at AAAI 2021 (7.7%) are on Architectures and efficiency, compared with 7.5% at AAAI 2020. They fall into 7 topics.
See AAAI 2021 in the interactive map
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
RNNs and normalization · 42 papers
- MiniSeg: An Extremely Minimum Network for Efficient COVID-19 Segmentation (122 citations)
- ALP-KD: Attention-Based Layer Projection for Knowledge Distillation (112 citations)
- Deep Unsupervised Image Hashing by Maximizing Bit Entropy (98 citations)
- Guiding Non-Autoregressive Neural Machine Translation Decoding with Reordering Information (68 citations)
- LRC-BERT: Latent-representation Contrastive Knowledge Distillation for Natural Language Understanding (55 citations)
- Quantum-inspired Neural Network for Conversational Emotion Recognition (55 citations)
- Decoupled and Memory-Reinforced Networks: Towards Effective Feature Learning for One-Step Person Search (46 citations)
- AutoDropout: Learning Dropout Patterns to Regularize Deep Networks (44 citations)
- Learning Accurate and Interpretable Decision Rule Sets from Neural Networks (41 citations)
- Training Binary Neural Network without Batch Normalization for Image Super-Resolution (39 citations)
- Shuffling Recurrent Neural Networks (37 citations)
- Neural-Symbolic Integration: A Compositional Perspective (35 citations)
and 30 more in the interactive map.
Attention and transformers · 36 papers
- Nyströmformer: A Nyström-based Algorithm for Approximating Self-Attention (342 citations)
- Two-Stream Convolution Augmented Transformer for Human Activity Recognition (228 citations)
- DialogXL: All-in-One XLNet for Multi-Party Conversation Emotion Recognition (227 citations)
- Entity Structure Within and Throughout: Modeling Mention Dependencies for Document-Level Relation Extraction (192 citations)
- Self-Attention Attribution: Interpreting Information Interactions Inside Transformer (186 citations)
- Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphs (155 citations)
- Noninvasive Self-attention for Side Information Fusion in Sequential Recommendation (144 citations)
- Stock Selection via Spatiotemporal Hypergraph Attention Network: A Learning to Rank Approach (143 citations)
- Contrastive Triple Extraction with Generative Transformer (129 citations)
- Dual Sparse Attention Network For Session-based Recommendation (121 citations)
- Bridging Towers of Multi-task Learning with a Gating Mechanism for Aspect-based Sentiment Analysis and Sequential Metaphor Identification (89 citations)
- Context-Guided BERT for Targeted Aspect-Based Sentiment Analysis (86 citations)
and 24 more in the interactive map.
CNN design · 13 papers
- Invertible Concept-based Explanations for CNN Models with Non-negative Concept Activation Vectors (60 citations)
- ACT: an Attentive Convolutional Transformer for Efficient Text Classification (39 citations)
- Explaining Convolutional Neural Networks through Attribution-Based Input Sampling and Block-Wise Feature Aggregation (34 citations)
- Linearly Replaceable Filters for Deep Network Channel Pruning (33 citations)
- PDO-eS2CNNs: Partial Differential Operator Based Equivariant Spherical CNNs (17 citations)
- Tightening Robustness Verification of Convolutional Neural Networks with Fine-Grained Linear Approximation (16 citations)
- Fast and Compact Bilinear Pooling by Shifted Random Maclaurin (14 citations)
- Compressing Deep Convolutional Neural Networks by Stacking Low-dimensional Binary Convolution Filters (11 citations)
- RT3D: Achieving Real-Time Execution of 3D Convolutional Neural Networks on Mobile Devices (9 citations)
- CNN Profiler on Polar Coordinate Images for Tropical Cyclone Structure Analysis (8 citations)
- Multi-Proxy Wasserstein Classifier for Image Classification (5 citations)
- Positions, Channels, and Layers: Fully Generalized Non-Local Network for Singer Identification (4 citations)
and 1 more in the interactive map.
Neural architecture search · 12 papers
- BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search (259 citations)
- MUFASA: Multimodal Fusion Architecture Search for Electronic Health Records (46 citations)
- Regional Attention with Architecture-Rebuilt 3D Network for RGB-D Gesture Recognition (33 citations)
- Rethinking Bi-Level Optimization in Neural Architecture Search: A Gibbs Sampling Perspective (23 citations)
- Searching for Machine Learning Pipelines Using a Context-Free Grammar (19 citations)
- Neural Architecture Search as Sparse Supernet (17 citations)
- Fitting the Search Space of Weight-sharing NAS with Graph Convolutional Networks (13 citations)
- Teacher Guided Neural Architecture Search for Face Recognition (10 citations)
- NASTransfer: Analyzing Architecture Transferability in Large Scale Neural Architecture Search (8 citations)
- OpEvo: An Evolutionary Method for Tensor Operator Optimization (5 citations)
- AdvantageNAS: Efficient Neural Architecture Search with Credit Assignment (3 citations)
- Unchain the Search Space with Hierarchical Differentiable Architecture Search (1 citation)
Pruning and efficient tuning · 11 papers
- OPQ: Compressing Deep Neural Networks with One-shot Pruning-Quantization (68 citations)
- TransTailor: Pruning the Pre-trained Model for Improved Transfer Learning (58 citations)
- DPFPS: Dynamic and Progressive Filter Pruning for Compressing Convolutional Neural Networks from Scratch (39 citations)
- Improving the Efficiency and Effectiveness for BERT-based Entity Resolution (39 citations)
- AutoLR: Layer-wise Pruning and Auto-tuning of Learning Rates in Fine-tuning of Deep Networks (26 citations)
- Winning Lottery Tickets in Deep Generative Models (25 citations)
- Provable Benefits of Overparameterization in Model Compression: From Double Descent to Pruning Neural Networks (21 citations)
- IsoBN: Fine-Tuning BERT with Isotropic Batch Normalization (18 citations)
- ROSITA: Refined BERT cOmpreSsion with InTegrAted techniques (15 citations)
- On the Optimal Efficiency of A* with Dominance Pruning (1 citation)
- Revisiting Dominance Pruning in Decoupled Search
Quantization · 8 papers
- FracBits: Mixed Precision Quantization via Fractional Bit-Widths (73 citations)
- Distribution Adaptive INT8 Quantization for Training CNNs (68 citations)
- Any-Precision Deep Neural Networks (41 citations)
- Post-training Quantization with Multiple Points: Mixed Precision without Mixed Precision (35 citations)
- TRQ: Ternary Neural Networks With Residual Quantization (32 citations)
- Scalable Verification of Quantized Neural Networks (30 citations)
- SA-BNN: State-Aware Binary Neural Network (16 citations)
- Optimizing Information Theory Based Bitwise Bottlenecks for Efficient Mixed-Precision Activation Quantization (2 citations)
Spiking networks · 6 papers
- Going Deeper With Directly-Trained Larger Spiking Neural Networks (504 citations)
- Temporal-Coded Deep Spiking Neural Network with Easy Training and Robust Performance (77 citations)
- Near Lossless Transfer Learning for Spiking Neural Networks (59 citations)
- Training Spiking Neural Networks with Accumulated Spiking Flow (50 citations)
- Strategy and Benchmark for Converting Deep Q-Networks to Event-Driven Spiking Neural Networks (43 citations)
- Deep Spiking Neural Network with Neural Oscillation and Spike-Phase Information (16 citations)
