NeurIPS 2018 accepted papers on Architectures and efficiency
96 of the 1,008 papers accepted at NeurIPS 2018 (9.5%) are on Architectures and efficiency, compared with 8.3% at NeurIPS 2017. They fall into 7 topics.
See NeurIPS 2018 in the interactive map
Submitting to NeurIPS? Checklist →
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
RNNs and normalization · 55 papers
- Efficient Neural Network Robustness Certification with General Activation Functions (332 citations)
- How Does Batch Normalization Help Optimization? (226 citations)
- Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs (203 citations)
- Learning to Optimize Tensor Programs (187 citations)
- Supervised autoencoders: Improving generalization performance with unsupervised regularizers (149 citations)
- ResNet with one-neuron hidden layers is a Universal Approximator (142 citations)
- Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures (102 citations)
- Norm matters: efficient and accurate normalization schemes in deep networks (100 citations)
- FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network (95 citations)
- Moonshine: Distilling with Cheap Convolutions (95 citations)
- Multi-Layered Gradient Boosting Decision Trees (88 citations)
- Neural Arithmetic Logic Units (77 citations)
and 43 more in the interactive map.
CNN design · 22 papers
- DropBlock: A regularization method for convolutional networks (509 citations)
- Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks (499 citations)
- An intriguing failing of convolutional neural networks and the CoordConv solution (169 citations)
- Task-Driven Convolutional Recurrent Models of the Visual System (137 citations)
- Frequency-Domain Dynamic Pruning for Convolutional Neural Networks (99 citations)
- Group Equivariant Capsule Networks (93 citations)
- Attention in Convolutional LSTM for Gesture Recognition (84 citations)
- Learning long-range spatial dependencies with horizontal gated recurrent units (80 citations)
- Faster Neural Networks Straight from JPEG (74 citations)
- Learning Versatile Filters for Efficient Convolutional Neural Networks (41 citations)
- Clebsch–Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network (36 citations)
- ChannelNets: Compact and Efficient Convolutional Neural Networks via Channel-Wise Convolutions (33 citations)
and 10 more in the interactive map.
Attention and transformers · 5 papers
- Layer-Wise Coordination between Encoder and Decoder for Neural Machine Translation (115 citations)
- Tree-to-tree Neural Networks for Program Translation (82 citations)
- Densely Connected Attention Propagation for Reading Comprehension (47 citations)
- Relational recurrent neural networks (26 citations)
- Blockwise Parallel Decoding for Deep Autoregressive Models (7 citations)
Quantization · 5 papers
- Training Deep Neural Networks with 8-bit Floating Point Numbers (202 citations)
- Scalable methods for 8-bit training of neural networks (197 citations)
- GradiVeQ: Vector Quantization for Bandwidth-Efficient Gradient Aggregation in Distributed CNN Training (48 citations)
- Heterogeneous Bitwidth Binarization in Convolutional Neural Networks (29 citations)
- HitNet: Hybrid Ternary Recurrent Neural Network (26 citations)
Pruning and efficient tuning · 4 papers
- Discrimination-aware Channel Pruning for Deep Neural Networks (375 citations)
- TETRIS: TilE-matching the TRemendous Irregular Sparsity (20 citations)
- A Simple Cache Model for Image Recognition (8 citations)
- Learning sparse neural networks via sensitivity-driven regularization (6 citations)
Spiking networks · 3 papers
Neural architecture search · 2 papers
- Neural Architecture Optimization (472 citations)
- Transfer Learning with Neural AutoML (41 citations)
