ICLR 2018 accepted papers on Architectures and efficiency
81 of the 337 papers accepted at ICLR 2018 (24.0%) are on Architectures and efficiency, compared with 28.8% at ICLR 2017. They fall into 6 topics.
See ICLR 2018 in the interactive map
Submitting to ICLR? Checklist →
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
RNNs and normalization · 44 papers
- Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training (1,018 citations)
- Non-Autoregressive Neural Machine Translation (507 citations)
- Regularizing and Optimizing LSTM Language Models (454 citations)
- Deep Active Learning for Named Entity Recognition (381 citations)
- Towards better understanding of gradient-based attribution methods for Deep Neural Networks (372 citations)
- Training and Inference with Integers in Deep Neural Networks (330 citations)
- Learning how to explain neural networks: PatternNet and PatternAttribution (291 citations)
- Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches (182 citations)
- Deep Complex Networks (167 citations)
- A Scalable Laplace Approximation for Neural Networks (157 citations)
- Compositional Attention Networks for Machine Reasoning (128 citations)
- Understanding Deep Neural Networks with Rectified Linear Units (124 citations)
and 32 more in the interactive map.
CNN design · 15 papers
- Matrix capsules with EM routing (726 citations)
- Depthwise Separable Convolutions for Neural Machine Translation (242 citations)
- Multi-Scale Dense Networks for Resource Efficient Image Classification (238 citations)
- Spherical CNNs (152 citations)
- Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers (149 citations)
- Wavelet Pooling for Convolutional Neural Networks (130 citations)
- Mixed Precision Training of Convolutional Neural Networks using Integer Operations (124 citations)
- i-RevNet: Deep Invertible Networks (57 citations)
- Efficient Sparse-Winograd Convolutional Neural Networks (47 citations)
- Polar Transformer Networks (30 citations)
- An image representation based convolutional network for DNA classification (28 citations)
- On the Expressive Power of Overlapping Architectures of Deep Learning (22 citations)
and 3 more in the interactive map.
Quantization · 9 papers
- Mixed Precision Training (999 citations)
- Model compression via distillation and quantization (252 citations)
- WRPN: Wide Reduced-Precision Networks (213 citations)
- Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy (148 citations)
- Loss-aware Weight Quantization of Deep Networks (91 citations)
- Alternating Multi-bit Quantization for Recurrent Neural Networks (89 citations)
- Variational Network Quantization (49 citations)
- Training wide residual networks for deployment using a single bit for each weight (41 citations)
- Adaptive Quantization of Neural Networks (36 citations)
Attention and transformers · 8 papers
- QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension (411 citations)
- FusionNet: Fusing via Fully-aware Attention with Application to Machine Comprehension (130 citations)
- DCN+: Mixed Objective And Deep Residual Coattention for Question Answering (103 citations)
- Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Modeling (77 citations)
- Generating Wikipedia by Summarizing Long Sequences (74 citations)
- Monotonic Chunkwise Attention (32 citations)
- Towards Neural Phrase-based Machine Translation (23 citations)
- Multi-Mention Learning for Reading Comprehension with Neural Cascades (3 citations)
Pruning and efficient tuning · 3 papers
Neural architecture search · 2 papers
- SMASH: One-Shot Model Architecture Search through HyperNetworks (386 citations)
- Hierarchical Representations for Efficient Architecture Search (226 citations)
