AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

Training Faster by Separating Modes of Variation in Batch-Normalized Models

Mahdi M. Kalayeh, Mubarak Shah

TPAMI 2019 · Architectures and efficiency › RNNs and normalization · approach: Batch normalization improvements · 59 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace BN in conv3 with mixture normalizationCIFAR CNN · CIFAR-10 (lr 0.01, wd 2e-5, 100 epochs)test accuracy (%): 87.95 -> 88.47Improved
replace BN in conv3 with mixture normalization at large learning rateCIFAR CNN · CIFAR-100 (lr 0.05, wd 1e-4)test accuracy (%): 36.80 -> 43.77Improved
replace BN in conv3 with mixture normalizationCIFAR CNN · CIFAR-100 (lr 0.01, wd 1e-4)test accuracy (%): 62.11 -> 62.20No clear effect
replace BN in conv3 with mixture normalizationCIFAR CNN · CIFAR-10 (lr 0.05, wd 1e-4)test accuracy (%): 77.35 -> 77.33No clear effect
increase EM iterations from 2 to 8 (ablation)MN in conv3 · CIFAR-100 CIFAR CNNtest accuracy (%): 62.29 -> 63.12Improved
increase EM iterations from 2 to 4 (ablation)MN in conv3 · CIFAR-100 CIFAR CNNtest accuracy (%): 62.29 -> 62.76Improved
apply MN to conv2 and conv3 instead of conv3 only (ablation)CIFAR CNN · CIFAR-100test accuracy (%): 62.29 -> 62.96Improved
use mixture normalization instead of BN (training steps to match BN accuracy)CIFAR CNN · CIFAR-100steps to 62.14%: 1.81 x 10^4 -> 1.07 x 10^4Improved
use mixture normalization in Inception-V3 (steps to 91.50%)Inception-V3 · CIFAR-10training steps: 7.34 x 10^4 -> 3.58 x 10^4Improved
apply MN to early reduction modules only (red1, red2)Inception-V3 · CIFAR-100test accuracy (%): 71.30 -> 70.43Hurt
increase mixture components from K=3 to K=5 (ablation)Inception-V3 MN · CIFAR-10test accuracy (%): 92.30 -> 92.25No clear effect
use mixture normalization at large learning rate 0.005Inception-V3 · CIFAR-100test accuracy (%): 52.63 -> 59.10Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy