Research map / Optimization / Distributed and compressed
Stabilized Proximal-Point Methods for Federated Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes S-DANE and Acc-S-DANE, federated proximal-point methods with stabilized prox-centers that match DANE's communication complexity with cheaper local computation, plus line-search adaptive variants.
Problem
Federated distributed convex optimization where DANE achieves the best non-accelerated communication complexity but requires local subproblems solved to high accuracy, costing extra local computation; existing accelerated methods either depend on the maximum second-order dissimilarity or do not matc
Approach
Uses a stabilized (hybrid projection-proximal style) auxiliary prox-center sequence updated by an extra gradient-type step, which relaxes the subproblem accuracy requirement; combined with Monteiro-Svaiter acceleration for an accelerated variant, and with line search to adapt the regularization without knowing the similarity constant.
Key results
- S-DANE matches DANE's deterministic communication complexity (delta D^2/epsilon for general convex) while reducing local gradient cost from sqrt-type dependence to sqrt(L/delta) per round for smooth functions vs DANE's log-dependent cost.
- Acc-S-DANE improves the strongly convex communication complexity from O~(delta/mu) to O~(sqrt(delta/mu)) and the general convex one from delta D^2/eps to sqrt(delta D^2/eps), claimed best among known distributed methods.
- On CIFAR-10 with ResNet-18, S-DANE (DL) reaches 90% accuracy within 50 communication rounds while other methods remain below 90% after 80 epochs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace DANE's prox-center with stabilized prox-centers | DANE · General convex and μ-strongly convex | # local gradient queries: \sqrt{\frac{L}{\delta}} \log \frac{L D^2}{\varepsilon} -> \sqrt{\frac{L}{\delta}}; \sqrt{\frac{L}{\delta}} \log\bigl( \f | Helped |
| accelerate S-DANE | S-DANE · General convex and μ-strongly convex | # communication rounds: \frac{\delta D^2}{\varepsilon} -> \sqrt{\frac{\delta D^2}{\varepsilon}}; \frac{\delta}{\mu} \log \frac{D^2}{\varepsilon} | Helped |
| use S-DANE (DL) for CIFAR10 classification | ResNet-18 · CIFAR10 | accuracy: 90% accuracy within 50 communication rounds | Helped |
| use adaptive λ with line search | S-DANE and Acc-S-DANE · Regularized logistic regression on ijcnn | convergence | Helped |
| reduce the number of sampled clients | Acc-S-DANE compared with S-DANE · Polyhedron feasibility problem with client sampling | convergence | Mixed |
Surprising findings
- The improvement of Acc-S-DANE over S-DANE gradually disappears as the number of sampled clients decreases in the partial participation experiments.
Limitations
- Assumes each local f_i is mu-strongly convex in all theorems, stronger than assuming only f is strongly convex.
- Theory is restricted to convex problems; non-convex extensions left to future work.
- Deep learning experiments are limited and effectiveness on Transformer models is unexplored.
Experimental setup
Datasets: ijcnn (LIBSVM) CIFAR-10 synthetic quadratic problems synthetic polyhedron feasibility problem
Benchmarks: CIFAR-10 ijcnn logistic regression synthetic quadratic minimization polyhedron feasibility
Metrics: communication rounds local gradient oracle calls objective suboptimality test accuracy
Baselines: DANE FedRed Acc-Extragradient AccSVRS Scaffold Scaffnew FedProx SONATA
Builds on: DANE Monteiro-Svaiter acceleration hybrid projection-proximal point method Catalyst
Closest papers by meaning
- On Convergence of FedProx: Local Dissimilarity Invariant Bounds, Non-smoothness and Beyond (NeurIPS 2022 · similarity 0.92)
- Federated Optimization with Doubly Regularized Drift Correction (ICML 2024 · similarity 0.91)
- Exploiting Similarity for Computation and Communication-Efficient Decentralized Optimization (ICML 2025 · similarity 0.88)
- Faster Rates For Federated Variational Inequalities (NeurIPS 2026 · similarity 0.88)
- Efficient Adaptive Federated Optimization (NeurIPS 2025 · similarity 0.88)
- Methods with Local Steps and Random Reshuffling for Generally Smooth Non-Convex Federated Optimization (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.