Research map / Optimization / Nonconvex and smooth optimization
Decentralized Projected Riemannian Stochastic Recursive Momentum Method for Nonconvex Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DPRSRM, a single-loop decentralized projected Riemannian stochastic recursive momentum method with gradient tracking for online optimization on compact submanifolds, claiming O(eps^-3/2) per-node SFO complexity with one consensus step per iteration.
Problem
Decentralized nonconvex optimization over a compact smooth submanifold (e.g., Stiefel manifold) in the online/streaming setting with stochastic first-order oracles, where existing methods have O(eps^-2) oracle complexity.
Approach
Combines a hybrid stochastic recursive momentum estimator (RSARAH/SRG-style, with clipping) with decentralized gradient tracking and projection-based single-step consensus on the manifold, yielding a single-loop variance-reduced method with O(1) gradient evaluations per iteration and no large-batch restarts.
Key results
- DPRSRM claims O(eps^-3/2) per-node SFO complexity versus at most O(eps^-2) for prior online decentralized Riemannian methods.
- In decentralized PCA experiments on synthetic and MNIST data, DPRSRM outperforms DRSGD and DPRGD in objective value, gradient norm, and distance to optimum.
- On decentralized low-rank matrix completion with a Ring graph, DPRSRM outperforms DRSGD.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DPRSRM instead of existing online decentralized stochastic manifold methods | Online decentralized Riemannian optimization · Online decentralized Riemannian optimization | oracle complexity: \mathcal{O}(\epsilon^{-2}) -> \mathcal{O}(\epsilon^{-3/2}) | Helped |
| Use DPRSRM instead of DRSGD and DRPGD | Decentralized PCA · PCA synthetic dataset, ER p=0.6 | objective function, gradient norm, and distance to optimum | Helped |
| Use DPRSRM instead of DRSGD and DRPGD | Decentralized PCA · Mnist dataset, ER p=0.3 | objective function, gradient norm, and distance to optimum | Helped |
| Use DPRSRM instead of DRSGD | Decentralized low-rank matrix completion · Low-rank matrix completion, Ring graph | performance | Helped |
| Change the network graph among Ring, ER p=0.3, and ER p=0.6 | Decentralized PCA · PCA synthetic dataset | performance excluding consensus error | No clear effect |
Limitations
- Theory relies on proximal smoothness, bounded gradients via clipping with B >= G_max, and a small step size constraint on the consensus neighborhood.
- Experiments are limited to PCA and low-rank matrix completion on relatively small problems; LRMC results are only in the supplement.
- Consensus error under stochastic gradients is affected by noise and needs smaller step sizes.
Experimental setup
Datasets: MNIST synthetic PCA data randomly generated low-rank matrix completion data
Benchmarks: MNIST PCA synthetic PCA synthetic low-rank matrix completion
Metrics: objective value gradient norm distance to optimal solution consensus error
Baselines: DRSGD DPRGD VRSGT
Builds on: STORM SARAH gradient tracking projected Riemannian gradient descent
Closest papers by meaning
- SERENA: A Unified Stochastic Recursive Variance Reduced Gradient Framework for Riemannian Non-Convex Optimization (ICML 2025 · similarity 0.91)
- Finite-Time Analysis of Stochastic Nonconvex Nonsmooth Optimization on the Riemannian Manifolds (NeurIPS 2025 · similarity 0.89)
- Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifold (ICLR 2024 · similarity 0.88)
- Decentralized Gradient-Free Methods for Stochastic Non-smooth Non-convex Optimization (AAAI 2024 · similarity 0.88)
- Efficient Optimization with Orthogonality Constraint: a Randomized Riemannian Submanifold Method (ICML 2025 · similarity 0.88)
- An Online Optimization Perspective on First-Order and Zero-Order Decentralized Nonsmooth Nonconvex Stochastic Optimization (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.