ICLR 2019 accepted papers on Architectures and efficiency
83 of the 502 papers accepted at ICLR 2019 (16.5%) are on Architectures and efficiency, compared with 24.0% at ICLR 2018. They fall into 6 topics.
See ICLR 2019 in the interactive map
Submitting to ICLR? Checklist →
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
RNNs and normalization · 45 papers
- Context-adaptive Entropy Model for End-to-end Optimized Image Compression (273 citations)
- Attention, Learn to Solve Routing Problems! (186 citations)
- Differentiable Learning-to-Normalize via Switchable Normalization (161 citations)
- AntisymmetricRNN: A Dynamical System View on Recurrent Neural Networks (119 citations)
- Fixup Initialization: Residual Learning Without Normalization (102 citations)
- The Unusual Effectiveness of Averaging in GAN Training (102 citations)
- Trellis Networks for Sequence Modeling (94 citations)
- Stable Recurrent Models (93 citations)
- Quaternion Recurrent Neural Networks (89 citations)
- How to train your MAML (81 citations)
- Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks (77 citations)
- Selfless Sequential Learning (75 citations)
and 33 more in the interactive map.
CNN design · 11 papers
- Dynamic Channel Pruning: Feature Boosting and Suppression (204 citations)
- Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet (141 citations)
- The Singular Values of Convolutional Layers (127 citations)
- Deep Convolutional Networks as shallow Gaussian Processes (98 citations)
- A Unified Theory of Early Visual Representations from Retina to Cortex through Anatomically Constrained Deep CNNs (88 citations)
- Big-Little Net: An Efficient Multi-Scale Feature Representation for Visual and Speech Recognition (83 citations)
- Spherical CNNs on Unstructured Grids (64 citations)
- Learning Implicitly Recurrent CNNs Through Parameter Sharing (41 citations)
- A rotation-equivariant convolutional neural network model of primary visual cortex (28 citations)
- RotDCF: Decomposition of Convolutional Filters for Rotation-Equivariant Deep Networks (12 citations)
- Human-level Protein Localization with Convolutional Neural Networks (10 citations)
Pruning and efficient tuning · 9 papers
- The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks (1,303 citations)
- Rethinking the Value of Network Pruning (958 citations)
- SNIP: SINGLE-SHOT NETWORK PRUNING BASED ON CONNECTION SENSITIVITY (559 citations)
- DELTA: DEEP LEARNING TRANSFER USING FEATURE MAP WITH ATTENTION FOR CONVOLUTIONAL NETWORKS (94 citations)
- An Empirical study of Binary Neural Networks' Optimisation (60 citations)
- K for the Price of 1: Parameter-efficient Multi-task and Transfer Learning (44 citations)
- Minimal Random Code Learning: Getting Bits Back from Compressed Model Parameters (19 citations)
- Efficient Augmentation via Data Subsampling (12 citations)
- Double Viterbi: Weight Encoding for High Compression Ratio and Fast On-Chip Reconstruction for Deep Neural Network (10 citations)
Attention and transformers · 8 papers
- Universal Transformers (382 citations)
- Pay Less Attention with Lightweight and Dynamic Convolutions (306 citations)
- Music Transformer: Generating Music with Long-Term Structure (305 citations)
- What do you learn from context? Probing for sentence structure in contextualized word representations (135 citations)
- On the Turing Completeness of Modern Neural Network Architectures (83 citations)
- Adaptive Input Representations for Neural Language Modeling (53 citations)
- Hyperbolic Attention Networks (47 citations)
- Posterior Attention Models for Sequence to Sequence Learning (25 citations)
Neural architecture search · 6 papers
- DARTS: Differentiable Architecture Search (1,354 citations)
- ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware (284 citations)
- SNAS: stochastic neural architecture search (276 citations)
- Graph HyperNetworks for Neural Architecture Search (114 citations)
- Efficient Multi-Objective Neural Architecture Search via Lamarckian Evolution (66 citations)
- Learnable Embedding Space for Efficient Neural Architecture Compression (27 citations)
