ICLR 2017 accepted papers on Architectures and efficiency
57 of the 198 papers accepted at ICLR 2017 (28.8%) are on Architectures and efficiency. They fall into 7 topics.
See ICLR 2017 in the interactive map
Submitting to ICLR? Checklist →
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
RNNs and normalization · 32 papers
- FractalNet: Ultra-Deep Neural Networks without Residuals (622 citations)
- Machine Comprehension Using Match-LSTM and Answer Pointer (456 citations)
- Quasi-Recurrent Neural Networks (325 citations)
- Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling (315 citations)
- Learning to Remember Rare Events (236 citations)
- Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks (213 citations)
- TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency (186 citations)
- Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations (168 citations)
- Tracking the World State with Recurrent Entity Networks (152 citations)
- Highway and Residual Networks learn Unrolled Iterative Estimation (150 citations)
- Paleo: A Performance Model for Deep Neural Networks (119 citations)
- Snapshot Ensembles: Train 1, Get M for Free (112 citations)
and 20 more in the interactive map.
CNN design · 8 papers
- Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer (1,535 citations)
- Designing Neural Network Architectures using Reinforcement Learning (412 citations)
- Faster CNNs with Direct Sparse Convolutions and Guided Pruning (133 citations)
- Do Deep Convolutional Nets Really Need to be Deep and Convolutional? (105 citations)
- Regularizing CNNs with Locally Constrained Decorrelations (97 citations)
- Inductive Bias of Deep Convolutional Networks through Pooling Geometry (40 citations)
- Steerable CNNs (21 citations)
- FILTER SHAPING FOR CONVOLUTIONAL NEURAL NETWORKS (17 citations)
Attention and transformers · 6 papers
- A STRUCTURED SELF-ATTENTIVE SENTENCE EMBEDDING (1,630 citations)
- Bidirectional Attention Flow for Machine Comprehension (1,592 citations)
- Pointer Sentinel Mixture Models (460 citations)
- Structured Attention Networks (98 citations)
- Latent Sequence Decompositions (36 citations)
- Frustratingly Short Attention Spans in Neural Language Modeling (10 citations)
Pruning and efficient tuning · 5 papers
- Pruning Convolutional Neural Networks for Resource Efficient Inference (1,321 citations)
- Pruning Filters for Efficient ConvNets (666 citations)
- DSD: Dense-Sparse-Dense Training for Deep Neural Networks (159 citations)
- Exploring Sparsity in Recurrent Neural Networks (138 citations)
- Soft Weight-Sharing for Neural Network Compression (79 citations)
Quantization · 4 papers
- Trained Ternary Quantization (836 citations)
- Incremental Network Quantization: Towards Lossless CNNs with Low-precision Weights (481 citations)
- Loss-aware Binarization of Deep Networks (167 citations)
- Towards the Limit of Network Quantization (60 citations)
Mixture of experts · 1 paper
Neural architecture search · 1 paper
- Neural Architecture Search with Reinforcement Learning (4,370 citations)
