AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

NNiT: Width-Agnostic Neural Network Generation with Structurally Aligned Weight Spaces

Jiwoo Kim, Swarajh Mehta, Hao-Lun Hsu, Hyunwoo Ryu, Yudong Liu, Miroslav Pajic

ICML 2026 · Architectures and efficiency › RNNs and normalization · approach: Theory of depth and width · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace flattened/zero-padded vectorization with patch tokenization of weightsNNiT vs D2NWG · PickCube-v1 unseen architecturessuccess rate: 59% -> 99%Improved
replace flattened/zero-padded vectorization with patch tokenization of weightsNNiT vs D2NWG · StackCubeEasy-v1 unseen architecturessuccess rate: 42% -> 86%Improved
patch tokenization vs D2NWG on the simple PushCube taskNNiT vs D2NWG · PushCube-v1 unseen architecturessuccess rate: 89% -> 100%Improved
use SANE row-wise slicing with global positional embeddings on mixed widthsSANE · seen and unseen architecturessuccess rate: 1% (PickCube seen), 0% (StackCubeEasy)Hurt
generate weights with a GHN with CNN decoder instead of SGD trainingtraining policy dataset · PickCube-v1, 35 seedsnearest-neighbor L2 distance: 41.73 -> 54.66Improved
generate weights with a GHN with CNN decoder instead of SGD trainingpolicy dataset · PickCube-v1, 35 seedsmean success rate: 98.9% -> 99.0%No clear effect
GHN with CNN decoder aligns weight space (vs SGD weights)weight space structure · PickCube-v1, 35 seedsqualitative alignment of weight magnitude heatmapsImproved
train jointly on architecture tokens and weight patches (MoNL) for joint generationNNiT · StackCubeEasy-v1success rate: 91% (seen conditional) -> 90% (joint)Mixed
permute weights to remove GHN alignment (ablation)NNiT · permutation ablationperformanceHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy