AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

On the Symmetries of Deep Learning Models and their Internal Representations

Charles Godfrey, Davis Brown, Tegan Emerson, Henry Kvinge

NeurIPS 2022 · Architectures and efficiency › RNNs and normalization · approach: Symmetry and equivariance in loss landscapes · 6 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
increase LeakyReLU negative slope from 0.1 to 1.0 (ablation)ResNet20 · CIFAR-10accuracy (%): 89.4 -> 41.8Hurt
increase LeakyReLU negative slope from 1e-4 to 0.1 (ablation)ResNet20 · CIFAR-10accuracy (%): 89.3 -> 89.4No clear effect
increase LeakyReLU negative slope to 0.9 (ablation)ResNet20 · CIFAR-10accuracy (%): 89.4 -> 73.0Hurt
use orthogonal instead of permutation/scaling invariance in ProcrustesMyrtle CNN · CIFAR-10 layer 3Procrustes similarity: 0.6208 -> 0.7724Mixed
remove ReLU on residual outputs (ablation)ResNet-50 · ImageNet network dissectioninterpretable units under rotationMixed
replace ResNet-50 with ConvNeXt (no residual activation) (ablation)ResNet-50 · ImageNet network dissectioninterpretable units under rotationMixed
use constant channel width 512 in Myrtle CNN (ablation)Myrtle CNN · CIFAR-10dissimilarity measuresNo clear effect
choose alpha by cross validation instead of fixed 0.1 (ablation)G_ReLU stitching · CIFAR-10stitching accuracyNo clear effect
l1-regularized stitching lambda 0.001-0.1 vs G_ReLU stitching (ablation)Myrtle CNN · CIFAR-10stitching penalty vs sparsityMixed
set l1 penalty lambda to 1 (ablation)l1 stitching · CIFAR-10optimization stabilityHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy