AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

Identity Matters in Deep Learning

Moritz Hardt, Tengyu Ma

ICLR 2017 · Architectures and efficiency › RNNs and normalization · approach: Normalization layer theory · 69 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use the all-convolutional residual architecture with ReLU and no batch normalization, dropout, or max pooling instead of All-CNNOurs · CIFAR10top-1 classification error: 7.25 -> 6.38Improved
Use the all-convolutional residual architecture with ReLU and no batch normalization, dropout, or max pooling instead of All-CNNOurs · CIFAR100top-1 classification error: 32.39 -> 24.64Improved
Use the all-convolutional residual architecture with ReLU and no batch normalization, dropout, or max pooling instead of All-CNNOurs · ImageNet ILSVRC 2012top-1 classification error: 41.2 -> 35.29Improved
Use the all-convolutional residual architecture instead of the original 34-layer counterpartOurs · ImageNet ILSVRC 2012top-1 classification error: 25.03% -> 35.29%Hurt
Use a fixed random projection for the last layer instead of training that layerOurs · CIFAR10 and CIFAR100test errorImproved
Initialize residual convolution weights with the smaller standard deviation 1/k^2cOurs · CIFAR10 and CIFAR100trainingImproved
Use batch normalization in a model of this sizeCIFAR10 · CIFAR10overfittingHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy