Research map / Architectures and efficiency / RNNs and normalization

Revisiting Mode Connectivity in Neural Networks with Bezier Surface

ICLR 2025 · Architectures and efficiency › RNNs and normalization · approach: Symmetry and equivariance in loss landscapes

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Extends mode connectivity from curves to Bezier surfaces in parameter space, connecting multiple well-trained neural networks through a low-loss surface.

Problem

Prior mode connectivity work connects pairs of trained networks via low-loss curves in parameter space; the connectivity structure over more than two models is less understood.

Approach

Generalizes mode connectivity from curves to surfaces by parameterizing a Bezier surface whose control points are trained so that the whole surface maintains nearly constant loss, connecting several trained networks at once.

Experimental setup

Builds on: mode connectivity Bezier curves

Closest papers by meaning

• Optimizing Mode Connectivity via Neuron Alignment (NeurIPS 2020 · similarity 0.89)
• Understanding Mode Connectivity via Parameter Space Symmetry (ICML 2025 · similarity 0.89)
• Input Space Mode Connectivity in Deep Neural Networks (ICLR 2025 · similarity 0.88)
• Generalized Linear Mode Connectivity for Transformers (NeurIPS 2025 · similarity 0.88)
• On Linear Mode Connectivity of Mixture-of-Experts Architectures (NeurIPS 2025 · similarity 0.86)
• Flat Channels to Infinity in Neural Loss Landscapes (NeurIPS 2025 · similarity 0.84)