Research map / Optimization / Nonconvex and smooth optimization
Provably Convergent Federated Trilevel Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AFTO, an asynchronous federated trilevel optimization method that builds hyper-polyhedral approximations from mu-cuts (valid for mu-weakly convex functions), with an O(1/epsilon^2) non-asymptotic iteration complexity and experiments on distributed robust hyperparameter optimization and domain adaptation.
Problem
Trilevel optimization is mostly studied in non-distributed settings (raising privacy concerns), and existing methods lack non-asymptotic convergence guarantees; cutting-plane methods for trilevel problems are limited to convex objectives.
Approach
Uses mu-cuts, a cutting-plane generalization derived from the first-order condition of mu-weakly convex functions, to build an outer hyper-polyhedral approximation of the nested lower levels, and solves the resulting problem with an asynchronous federated algorithm where the master updates after S of N workers respond.
Key results
- AFTO reaches MSE 0.3130 on Boston vs 0.3243 for ADBO and 0.3509 for FEDNEST (test data with Gaussian noise).
- On domain adaptation with SVHN/MNIST, AFTO converges faster than SFTO with a maximum acceleration of approximately 80%.
- Theoretical iteration complexity to reach an epsilon-stationary point is O(1/epsilon^2).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AFTO instead of ADBO | Distributed robust hyperparameter optimization · Diabetes | MSE: 0.5284 ± 0.0074 -> 0.5124 ± 0.0068 | Helped |
| Use AFTO instead of ADBO | Distributed robust hyperparameter optimization · Boston | MSE: 0.3243 ± 0.0046 -> 0.3130 ± 0.0037 | Helped |
| Use AFTO instead of ADBO | Distributed robust hyperparameter optimization · Red-wine | MSE: 0.0336 ± 0.0018 -> 0.0321 ± 0.0026 | Helped |
| Use AFTO instead of FEDNEST | Distributed robust hyperparameter optimization · White-wine | MSE: 0.0268 ± 0.0010 -> 0.0248 ± 0.0021 | Helped |
| Update the master after receiving updates from a subset of workers rather than waiting for all workers | AFTO · Distributed robust hyperparameter optimization and domain adaptation | convergence rate | Helped |
| Use AFTO instead of SFTO | Distributed domain adaptation · MNIST and SVHN digit recognition | convergence rate: maximum acceleration of approximately 80$\%$ | Helped |
Limitations
- Convergence analysis relies on bounded-variable and bounded-change assumptions (Assumptions on bounded variables and staleness tau).
- Subproblem solutions are approximated by K communication rounds, so the polyhedral relaxation is only approximate.
- Experiments are limited to small-scale regression and digit recognition tasks with LeNet-5/MLP models.
Experimental setup
Datasets: Diabetes Boston Red-wine quality White-wine quality MNIST SVHN
Benchmarks: Diabetes Boston Red-wine quality White-wine quality MNIST SVHN
Metrics: mean squared error test accuracy test loss running time to convergence
Baselines: SFTO ADBO FEDNEST
Builds on: bilevel federated optimization (ADBO, FEDNEST) cutting plane methods for convex optimization polyhedral approximation
Closest papers by meaning
- DTZO: Distributed Trilevel Zeroth Order Learning with Provable Non-Asymptotic Convergence (ICML 2025 · similarity 0.88)
- Moreau Envelope for Nonconvex Bi-Level Optimization: A Single-Loop and Hessian-Free Solution Strategy (ICML 2024 · similarity 0.85)
- Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity (NeurIPS 2026 · similarity 0.84)
- Stabilized Proximal-Point Methods for Federated Optimization (NeurIPS 2024 · similarity 0.84)
- Federated Optimization with Doubly Regularized Drift Correction (ICML 2024 · similarity 0.83)
- A Doubly Recursive Stochastic Compositional Gradient Descent Method for Federated Multi-Level Compositional Optimization (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.