Research map / Optimization / Distributed and compressed
Riemannian Diffusion Adaptation for Distributed Optimization on Manifolds
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Riemannian diffusion adaptation, an intrinsic adapt-then-combine decentralized R-SGD strategy on general manifolds with convergence analysis, applied to online distributed PCA on the Grassmannian and GMM inference on a product manifold of SPD matrices.
Problem
Decentralized streaming optimization over Riemannian manifolds seeking network consensus, where existing methods are extrinsic, need convexity or embeddings, or are inefficient and lack theory.
Approach
An intrinsic diffusion strategy: each agent takes a Riemannian SGD step with constant step size (adaptation), then combines neighbors' intermediate estimates via exponential-map-based consensus gradient step in the tangent space (combination), with non-asymptotic analysis using a curvature-dependent Lyapunov function.
Key results
- On synthetic PCA, the Riemannian diffusion method achieves lower MSD than the non-cooperative and DRSGD baselines, with the centralized solution lowest.
- On synthetic GMM inference, the proposed method outperforms the non-cooperative algorithm and ECGMM in ALL difference, with centralized best.
- Theory shows network agreement after O(mu^-1) iterations with E[P] <= O(mu^2), and a non-asymptotic convergence bound for the streaming averaged iterate under geodesic convexity.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Riemannian diffusion instead of non-cooperative local updates | Distributed PCA · Synthetic data | MSD | Helped |
| use Riemannian diffusion instead of DRSGD | Distributed PCA · Synthetic data | MSD | Helped |
| use Riemannian diffusion instead of non-cooperative local updates | Distributed PCA · MNIST | MSD | Helped |
| use Riemannian diffusion instead of DRSGD | Distributed PCA · MNIST | MSD | Helped |
| use a centralized strategy with access to data from all agents instead of Riemannian diffusion | Distributed PCA · Synthetic data | MSD | Helped |
| use a centralized strategy with access to data from all agents instead of Riemannian diffusion | Distributed PCA · MNIST | MSD | Helped |
| use Riemannian diffusion instead of non-cooperative local updates | GMM inference · Synthetic data | ALL difference (ALL* − ALL) | Helped |
| use Riemannian diffusion instead of ECGMM | GMM inference · Synthetic data | ALL difference (ALL* − ALL) | Helped |
| use Riemannian diffusion instead of non-cooperative local updates | GMM inference · MNIST | ALL difference (ALL* − ALL) | Helped |
| use Riemannian diffusion instead of ECGMM | GMM inference · MNIST | ALL difference (ALL* − ALL) | Helped |
| use a centralized strategy with access to data from all agents instead of Riemannian diffusion | GMM inference · Synthetic data | ALL difference (ALL* − ALL) | Helped |
| use a centralized strategy with access to data from all agents instead of Riemannian diffusion | GMM inference · MNIST | ALL difference (ALL* − ALL) | Helped |
Surprising findings
- The GMM log-likelihood is not geodesically convex for multiple Gaussians, yet the algorithm still performs well empirically.
Limitations
- Theoretical results rely on the exponential map, which can be computationally heavy.
- Assumptions such as geodesic convexity and bounded curvature/diameter may not hold for GMM inference.
- Centralized solution still outperforms the decentralized method in MSD and ALL.
Experimental setup
Datasets: MNIST synthetic Gaussian data
Benchmarks: MNIST synthetic Gaussian data
Metrics: mean square deviation (MSD) average log-likelihood (ALL) difference
Baselines: Riemannian non-cooperative Riemannian centralized DRSGD ECGMM
Builds on: diffusion adaptation R-SGD (Bonnabel 2013) geodesic consensus (Tron et al. 2012) Riemannian SGD non-asymptotic analysis (Zhang & Sra 2016)
Closest papers by meaning
- Decentralized Projected Riemannian Stochastic Recursive Momentum Method for Nonconvex Optimization (AAAI 2025 · similarity 0.87)
- Achieving Linear Convergence with Parameter-Free Algorithms in Decentralized Optimization (NeurIPS 2024 · similarity 0.86)
- Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifold (ICLR 2024 · similarity 0.86)
- Nonconvex Federated Learning on Compact Smooth Submanifolds With Heterogeneous Data (NeurIPS 2024 · similarity 0.85)
- Generative Modeling on Manifolds Through Mixture of Riemannian Diffusion Processes (ICML 2024 · similarity 0.85)
- Faster Adaptive Decentralized Learning Algorithms (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.