AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

The Sign Code: The Hidden Binary Nature of Deep Networks

Niclas A Göring, Shuofeng Zhang, Roi Holtzman, yoonsoo nam, Ard Louis

NeurIPS 2026 · Architectures and efficiency › RNNs and normalization · approach: Mechanistic circuit interpretability · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Investigates whether classification in overparameterized vision networks depends more on the binary sign code of neuron outputs or on their magnitudes.

Problem

Whether the decision of trained deep vision classifiers depends more on the sign pattern of neuron outputs (sign code) or on their magnitudes.

Approach

Defines the sign code as the set of all neuron signs in a trained network and argues that, for overparameterized vision classifiers with sign-preserving activations, classification is overwhelmingly determined by the sign code rather than magnitudes.

Key results

Measured effects reported in the paper

What was changedWhereMeasuredEffect
increase network depth and width (ablation)FFNN/ResNet/ViT, SiLU · CIFAR-10sign gap: negligible (< 0.03) once both largeImproved
flip 20% of neuron signs (ablation)ResNet-18 and FFNN · CIFAR-10 / SVHNaccuracy: to ~10% (chance)Hurt
replace every magnitude with random values while enforcing clean sign code (ablation)ResNet-18 · CIFAR-10accuracy: 93-95% retainedNo clear effect
replace every magnitude with random values while enforcing clean sign code (ablation)FFNN with ReLU/GELU/SiLU · SVHNaccuracy: drop ~36 / ~34 / ~32 ppHurt
replace every magnitude with random values while enforcing clean sign code (ablation)FFNN with tanh · SVHNaccuracy: drop ~2 ppNo clear effect
add within-class sign-code repulsion regularizer (ablation)SiLU network · CIFAR-10test accuracy: train 97%, test falls to ~40%Hurt
add within-class sign-code repulsion regularizer (ablation)tanh network · classificationtest accuracyNo clear effect
use only last-layer sign code with kNN instead of full modeloverparameterized networks · MNIST/CIFAR-10/CIFAR-100, 540 modelstest accuracyNo clear effect
use nearest class mean on features (neural collapse test) instead of full modeloverparameterized networks · CIFAR-10test accuracy: NCM peaks at 60-70% vs ~80% full modelHurt
use depth 1 network with width 4096 (ablation)FFNN · CIFAR-10gate-domination score Z1: Z1 < 0.23No clear effect

Surprising findings

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy