Research map / Optimization / Implicit bias and generalization
Git Re-Basin: Merging Models modulo Permutation Symmetries
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Align independently trained weights with weight matching rather than OT-Fusion | VGG11 · CIFAR-10 | test accuracy: 85.98% -> 86.57% | Helped |
| Align independently trained weights with weight matching rather than OT-Fusion | VGG11 · CIFAR-10 | run-time: 2.86s -> 0.64s | Helped |
| Use weight matching rather than OT-Fusion | ResNet50 · ImageNet | top-1 accuracy: 1.38% -> 51.01% | Helped |
| Merge five separately trained models with MergeMany | MLP · MNIST | test loss: 0.1108 -> 0.0727 | Helped |
| Merge five separately trained models with MergeMany | MLP · MNIST | test accuracy: 0.9871 -> 0.9831 | Hurt |
| Increase model width (ablation) | VGG-16 and ResNet20 · CIFAR-10 | loss barrier | Helped |
| Train models longer (ablation) | MLPs · MNIST and CIFAR-10 | loss barrier | Helped |
| Limit weight matching to a single pass over network layers (ablation) | ResNet50 · ImageNet | top-1 accuracy: 51.01% -> ~7% | Hurt |
| Initialize STE matching from random weights instead of the reference model (ablation) | STE matching · Loss interpolation experiments | solution quality | Hurt |
| Use naïve interpolation without permutation alignment (ablation) | Independently trained models · MNIST, CIFAR-10, and ImageNet | loss barrier | Hurt |
Closest papers by meaning
- Linear Mode Connectivity between Multiple Models modulo Permutation Symmetries (ICML 2025 · similarity 0.89)
- Generalized Linear Mode Connectivity for Transformers (NeurIPS 2025 · similarity 0.88)
- $C^2M^3$: Cycle-Consistent Multi-Model Merging (NeurIPS 2024 · similarity 0.85)
- Analysis of Linear Mode Connectivity via Permutation-Based Weight Matching: With Insights into Other Permutation Search Methods (ICLR 2025 · similarity 0.85)
- A Tale of Two Symmetries: Exploring the Loss Landscape of Equivariant Models (NeurIPS 2025 · similarity 0.85)
- On Linear Mode Connectivity of Mixture-of-Experts Architectures (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.