Research map / Learning theory and algorithms / Optimal transport
Towards Understanding Gradient Dynamics of the Sliced-Wasserstein Distance via Critical Point Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Theoretical analysis of critical points of the sliced-Wasserstein objective (squared SW_2 to a target), introducing Lagrangian critical points and proving that stable critical points cannot contain segments in dimension 2, with gradient-descent experiments.
Problem
Understanding the optimization dynamics and critical points of the sliced-Wasserstein distance when used as an objective for approximating a target measure with particles, including whether bad (lower-dimensional) critical points are stable.
Approach
Defines Lagrangian and barycentric Lagrangian critical points of SW_2^2 for measures, shows their equivalence under compact support and atomless assumptions, constructs explicit lower-dimensional critical points via symmetry, and proves that measures containing a segment are unstable in d=2.
Key results
- Measures containing a horizontal segment are shown to be unstable for SW_2^2 in d=2 when the target has bounded projection densities: SW_2^2(mu^t, rho) <= SW_2^2(mu, rho) - C t^2 for any C>0 near t=0.
- Explicit barycentric Lagrangian critical points are given: a segment-supported measure for the 2D sliced-uniform target and a Gaussian-axis measure for the standard Gaussian in dimension d>1.
- Gradient descent with step size near Nd (d=2) gives a large initial loss decrease, while step sizes above 2Nd diverge.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Perturb the segment by alternating points between e₂ and −e₂ | SW²₂ objective · Normal target | SW²₂ loss under perturbation | Helped |
| Perturb the segment by alternating points between e₂ and −e₂ | SW²₂ objective · Sliced-uniform target | SW²₂ loss under perturbation | Helped |
| Perturb the segment by alternating points between e₂ and −e₂ | SW²₂ objective · Uniform shell target | SW²₂ loss under perturbation | Helped |
| Perturb the segment in a point cloud with a more complex shape, leaving other points unchanged | SW²₂ objective · Point cloud containing a horizontal segment | SW²₂ loss under perturbation | Helped |
| Use step sizes close to λ = dN | SW²₂ gradient descent · Normal and sliced-uniform targets | Loss decrease in the first iterations | Helped |
| Use lower step sizes | SW²₂ gradient descent · Normal and sliced-uniform targets | Convergence speed | Hurt |
| Use step sizes larger than 2dN | SW²₂ gradient descent · Normal and sliced-uniform targets | Convergence | Hurt |
Surprising findings
- Stable critical points of SW_2^2 cannot concentrate on segments, though non-global critical points exist for full-dimensional targets.
Limitations
- Instability result only holds in dimension d=2 and uses a perturbation outside the Lagrangian framework.
- Most results require the target to be atomless, limiting application to discrete targets common in machine learning.
Experimental setup
Metrics: SW_2^2 value along perturbations SW_2^2 loss convergence under gradient descent
Builds on: Sliced-Wasserstein distance Lagrangian critical points of Wasserstein distance Bonnotte's SW differentiability results Lloyd's algorithm for Wasserstein (Power cells)
Closest papers by meaning
- Sliced Gromov-Wasserstein (NeurIPS 2019 · similarity 0.91)
- Tree-Sliced Variants of Wasserstein Distances (NeurIPS 2019 · similarity 0.90)
- Tree-Sliced Wasserstein Distance with Nonlinear Projection (ICML 2025 · similarity 0.89)
- Generalized Sliced Wasserstein Distances (NeurIPS 2019 · similarity 0.89)
- Statistical and Computational Guarantees of Kernel Max-Sliced Wasserstein Distances (ICML 2025 · similarity 0.89)
- Tree-Sliced Wasserstein Distance: A Geometric Perspective (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.