Research map / Architectures and efficiency / RNNs and normalization
Beyond Structural Symmetries: Linear Mode Connectivity via Neuron Identifiability
Vincent Bürgin, Daniel Herbst, Ya-Wei Eileen Lin, Stefanie Jegelka
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase the fixed-weight scale sigma_F in W-asymmetric networks (center-dominated regime) (ablation) | W-MLP and W-ResNet · MNIST / CIFAR-10 unaligned LMC | unaligned LMC barrier | Improved |
| break structural symmetry only (W-asymmetric with sigma_F=0) (ablation) | W-MLP · MNIST unaligned LMC | LMC barrier | No clear effect |
| align models by activation matching before interpolation | syre-ResNet · CIFAR-10 LMC | LMC barrier | Improved |
| align models by activation matching before interpolation | W-asym. architectures with sigma_F=0 · MNIST / CIFAR-10 LMC | LMC barrier | Improved |
| increase sigma_F so identity permutation nearly matches optimal activation-matching permutation (ablation) | W-MLP and W-ResNet · MNIST / CIFAR-10 activation matching objective | activation matching objective | Improved |
| increase intrinsic data dimension k from 2 to 32 (sigma_F=1) (ablation) | W-MLP on Gaussian mixture data · unaligned LMC midpoint train accuracy drop | accuracy drop (points): 46.4 -> 15.7 -> 6.1 for k=2, 8, 32 | Improved |
| increase input subspace coherence nu(U) in the F=0 setting (ablation) | W-asym MLP (D~Bernoulli(0.15)) · synthetic ReLU(a+b) regression | LMC midpoint barrier | Improved |
| use random mask D alone (without large fixed F) as symmetry breaker on incoherent data | W-asymmetric layers · theory (Thm 4.4) | effective symmetry breaking | No clear effect |
| fix biases only (F varies in one intrinsic dimension) | symmetry breaking via fixed biases · theory discussion | effective symmetry breaking | No clear effect |
Closest papers by meaning
- Parameter symmetries determine representational geometry in overparameterized nonlinear networks (NeurIPS 2026 · similarity 0.89)
- Generalized Linear Mode Connectivity for Transformers (NeurIPS 2025 · similarity 0.88)
- Understanding Mode Connectivity via Parameter Space Symmetry (ICML 2025 · similarity 0.87)
- Isotropic Activation Functions Enable Deindividuated Neurons and Adaptive Topologies (NeurIPS 2026 · similarity 0.87)
- The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof (NeurIPS 2024 · similarity 0.87)
- Beyond ICA: Identifiability by Symmetry Breaking (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
