AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

Low-dimensional topology of deep neural networks

Junyu Ren, Lek-heng Lim

ICML 2026 · Architectures and efficiency › RNNs and normalization · approach: Symmetry and equivariance in loss landscapes · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace ReLU with nonmonotonic GELU (ablation)width-3 feedforward, depth 8 · Hopf link classificationheld-out accuracy (%): 63.1 -> 91.1Improved
replace ReLU with nonmonotonic GELU (ablation)width-3 feedforward, depth 3 · Hopf link classificationheld-out accuracy (%): 84.3 -> 89.3Improved
replace ReLU with nonmonotonic GELU (ablation)width-3 feedforward, depth 20 · Hopf link classificationheld-out accuracy (%): 50.3 -> 52.8No clear effect
replace ReLU with GELU: best-seed ceiling (ablation)width-3 feedforward, depth 5 · Hopf link classificationmax accuracy over seeds (%): 92.5 -> 100.0Improved
add skip connections (ReLU ResNet) (ablation)width-3 ReLU network, depth 5 · Hopf link classificationheld-out accuracy (%): 74.3 -> 98.5Improved
add skip connections (ReLU ResNet) (ablation)width-3 ReLU network, depth 3 · Hopf link classificationheld-out accuracy (%): 83.9 -> 97.5Improved
add skip connections (ablation)width-5 ReLU, linked S2 pair k=1 · Linked S2 x S2 in R^5best test accuracy (%): 98.6 -> 100.0Improved
add skip connections (ablation)width-5 ReLU, k=10 linked copies · Linked S2 x S2 in R^5best test accuracy (%): 87.9 -> 85.4Hurt
replace ReLU with GELU (ablation)width-5 depth-5 network, k=50 · Linked S2 x S2 in R^5best test accuracy (%): 80.2 -> 84.8Improved
replace ReLU with Swish (ablation)width-5 depth-5 network, k=10 · Linked S2 x S2 in R^5best test accuracy (%): 87.9 -> 91.6Improved
use nonmonotonic activation (GELU) instead of ReLU (ablation)L8 no-skip CNN · CIFAR-10 deer-dog (linked pair)accuracy (%): 89.6 -> 90.8Improved
use nonmonotonic activation (GELU) instead of ReLU (ablation)L8 no-skip CNN · CIFAR-10 frog-ship (unlinked control)accuracy (%): 97.8 -> 97.9No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy