ICML 2019 accepted papers on Architectures and efficiency
56 of the 774 papers accepted at ICML 2019 (7.2%) are on Architectures and efficiency, compared with 9.5% at ICML 2018. They fall into 6 topics.
See ICML 2019 in the interactive map
Submitting to ICML? Checklist →
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
RNNs and normalization · 25 papers
- Training Neural Networks with Local Error Signals (131 citations)
- On The Power of Curriculum Learning in Training Deep Networks (128 citations)
- Complexity of Linear Regions in Deep Networks (102 citations)
- Shallow-Deep Networks: Understanding and Mitigating Network Overthinking (98 citations)
- On the Impact of the Activation function on Deep Neural Networks Training (67 citations)
- Ithemal: Accurate, Portable and Fast Basic Block Throughput Estimation using Deep Neural Networks (50 citations)
- SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver (43 citations)
- Traditional and Heavy Tailed Self Regularization in Neural Network Models (38 citations)
- LIT: Learned Intermediate Representation Training for Model Compression (34 citations)
- Adaptive Neural Trees (23 citations)
- State-Regularized Recurrent Neural Networks (23 citations)
- A fully differentiable beam search decoder (21 citations)
and 13 more in the interactive map.
CNN design · 13 papers
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks (4,992 citations)
- Making Convolutional Networks Shift-Invariant Again (441 citations)
- Gauge Equivariant Convolutional Networks and the Icosahedral CNN (165 citations)
- Approximated Oracle Filter Pruning for Destructive CNN Width Optimization (45 citations)
- Equivariant Transformer Networks (41 citations)
- LegoNet: Efficient Convolutional Neural Networks with Lego Filters (35 citations)
- IMEXnet A Forward Stable Deep Neural Network (28 citations)
- Greedy Layerwise Learning Can Scale To ImageNet (24 citations)
- Understanding Geometry of Encoder-Decoder CNNs (23 citations)
- Band-limited Training and Inference for Convolutional Neural Networks (21 citations)
- Memory-Optimal Direct Convolutions for Maximizing Classification Accuracy in Embedded Applications (17 citations)
- Training CNNs with Selective Allocation of Channels (12 citations)
and 1 more in the interactive map.
Attention and transformers · 7 papers
- Insertion Transformer: Flexible Sequence Generation via Insertion Operations (154 citations)
- Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks (150 citations)
- Efficient Training of BERT by Progressively Stacking (66 citations)
- Empirical Analysis of Beam Search Performance Degradation in Neural Sequence Models (42 citations)
- A Statistical Investigation of Long Memory in Language and Music (7 citations)
- Area Attention (4 citations)
- Deep Residual Output Layers for Neural Language Generation (4 citations)
Pruning and efficient tuning · 6 papers
- Parameter-Efficient Transfer Learning for NLP (141 citations)
- Parameter efficient training of deep convolutional neural networks by dynamic sparse reparameterization (119 citations)
- BERT and PALs: Projected Attention Layers for Efficient Adaptation in Multi-Task Learning (109 citations)
- Collaborative Channel Pruning for Deep Networks (85 citations)
- EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasis (30 citations)
- Rate Distortion For Model Compression:From Theory To Practice (19 citations)
Neural architecture search · 3 papers
- NAS-Bench-101: Towards Reproducible Neural Architecture Search (243 citations)
- The Evolved Transformer (185 citations)
- BayesNAS: A Bayesian Approach for Neural Architecture Search (85 citations)
