Research map / Optimization / Nonconvex and smooth optimization
Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifold
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DRCGD, a decentralized Riemannian conjugate gradient method over the Stiefel manifold that replaces retractions and vector transports with projections, with global convergence under an extended assumption.
Problem
Decentralized optimization of smooth, possibly non-convex local functions over the Stiefel manifold, where existing decentralized Riemannian methods use costly retractions/vector transports or lack conjugate-gradient variants.
Approach
Decentralized Fletcher-Reeves-type Riemannian conjugate gradient in which the consensus-averaged search directions are projected onto the current tangent space instead of being transported by a vector transport, and iterates are updated by projection onto the manifold instead of retraction.
Key results
- On synthetic eigenvector problems, DRCGD converges faster than DPRGD for n=16 and n=32 agents on a ring graph with t=1.
- DRCGD improves with more consensus steps t, while DRDGD and DPRGD are largely insensitive to t.
- On MNIST with n=20 agents (ring graph, single-step consensus), DRCGD converges much faster than DRDGD and DPRGD.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DRCGD instead of DPRGD | Decentralized eigenvector problem · Synthetic data, Ring network, n=16 and n=32 | convergence speed | Helped |
| Use DRCGD instead of DRDGD or DPRGD | Decentralized eigenvector problem · Synthetic data, different network graphs | convergence speed | Helped |
| Increase communication rounds t | DRCGD · Synthetic data, Ring network, n=16 | performance | Helped |
| Increase communication steps t | DRDGD · Synthetic data, Ring network, n=16 | performance | No clear effect |
| Increase communication steps t | DPRGD · Synthetic data, Ring network, n=16 | performance | No clear effect |
| Use an ER network instead of a Ring network | DRCGD · Synthetic data, t=10, n=16 | final solution quality | Helped |
| Increase ER edge probability from p=0.3 to p=0.6 | DRCGD · Synthetic data, t=10, n=16 | performance | Helped |
| Use DRCGD instead of DRDGD or DPRGD | Decentralized eigenvector problem · MNIST, Ring network, n=20 | convergence speed | Helped |
| Increase step size parameter hat-alpha | DRCGD, DRDGD, and DPRGD · MNIST, Ring network, n=20 | convergence speed | Helped |
Surprising findings
- DRDGD and DPRGD performance is nearly unchanged across consensus step counts, while DRCGD benefits from more consensus steps.
Limitations
- Global convergence relies on an extended assumption on the projected search-direction inner product that is not proven from basic assumptions.
- Requires bounded beta (||beta|| <= C) and step-size conditions for the analysis.
- Experiments are limited to eigenvector problems on small-scale synthetic and MNIST data with a CPU-only setup.
Experimental setup
Datasets: synthetic Gaussian data MNIST
Benchmarks: synthetic decentralized eigenvector problem MNIST eigenvector problem
Metrics: consensus error gradient norm objective gap distance to global optimum d_s
Baselines: DRDGD DPRGD
Builds on: Riemannian conjugate gradient decentralized projected Riemannian gradient descent decentralized Riemannian gradient descent Fletcher-Reeves
Closest papers by meaning
- Decentralized Projected Riemannian Stochastic Recursive Momentum Method for Nonconvex Optimization (AAAI 2025 · similarity 0.88)
- SERENA: A Unified Stochastic Recursive Variance Reduced Gradient Framework for Riemannian Non-Convex Optimization (ICML 2025 · similarity 0.87)
- Convergence and Complexity Guarantee for Inexact First-order Riemannian Optimization Algorithms (ICML 2024 · similarity 0.86)
- Finite-Time Analysis of Stochastic Nonconvex Nonsmooth Optimization on the Riemannian Manifolds (NeurIPS 2025 · similarity 0.86)
- Riemannian Diffusion Adaptation for Distributed Optimization on Manifolds (ICML 2025 · similarity 0.86)
- Riemannian coordinate descent algorithms on matrix manifolds (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.