Research map / Learning theory and algorithms / Prediction and decision losses
Optimizing Mode Connectivity via Neuron Alignment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| align neurons of second endpoint (correlation of post-activations) before learning Bezier curve | mode connecting curve, TinyTen · Tiny ImageNet test | min accuracy along curve: 30.0 -> 33.7 | Helped |
| add neuron alignment before curve finding | mode connecting curve, ResNet32 · CIFAR10 test | min accuracy along curve: 89.1 -> 92.2 | Helped |
| add neuron alignment before curve finding | mode connecting curve, GoogLeNet · CIFAR100 test | min accuracy along curve: 69.8 -> 72.9 | Helped |
| add neuron alignment before curve finding | mode connecting curve, GoogLeNet · CIFAR10 test | avg accuracy along curve: 93.3 -> 93.4 | No clear effect |
| add neuron alignment before curve finding | mode connecting curve, TinyTen · Tiny ImageNet test | avg accuracy along curve: 32.5 -> 34.4 | Helped |
| run PAM from aligned permutation instead of plain alignment (ablation) | Aligned curve, TinyTen · CIFAR100 test | avg accuracy along curve: 58.7 -> 58.8 | No clear effect |
| run PAM from identity initialization instead of alignment (ablation) | curve finding, TinyTen · CIFAR100 test | avg accuracy along curve: 58.7 -> 57.3 | Hurt |
| run PAM from identity init versus no permutation optimization (ablation) | Unaligned curve, TinyTen · CIFAR10 test | min accuracy along curve: 82.8 -> 84.0 | Helped |
| align post-activations by correlation instead of pre-activations / L2 distance (ablation) | alignment technique, TinyTen · CIFAR10/CIFAR100/Tiny ImageNet | test accuracy along curve | Helped |
| add neuron alignment when connecting adversarially robust models | robust curve, ResNet32 CIFAR100 · CIFAR100 clean and robust accuracy | midpoint accuracy vs endpoints: +5.3% clean, +1.3% robust | Helped |
| add neuron alignment | curve training speed, TinyTen CIFAR100 · CIFAR100 train accuracy | epochs to converge: 100 epochs less | Helped |
Closest papers by meaning
- Generalized Linear Mode Connectivity for Transformers (NeurIPS 2025 · similarity 0.92)
- Understanding Mode Connectivity via Parameter Space Symmetry (ICML 2025 · similarity 0.90)
- On Linear Mode Connectivity of Mixture-of-Experts Architectures (NeurIPS 2025 · similarity 0.88)
- Input Space Mode Connectivity in Deep Neural Networks (ICLR 2025 · similarity 0.87)
- Exploring and Exploiting the Asymmetric Valley of Deep Neural Networks (NeurIPS 2024 · similarity 0.87)
- The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.