Research map / Federated learning and privacy / Communication-efficient FL

A Federated Stochastic Multi-level Compositional Minimax Algorithm for Deep AUC Maximization

ICML 2024 · Federated learning and privacy › Communication-efficient FL · approach: Federated optimization algorithms

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LocalSMCGDAM, a federated stochastic multi-level compositional minimax algorithm for deep AUC maximization that uses multiple inner gradient steps with STORM-style estimators and momentum, with level-independent convergence analysis.

Problem

Federated deep AUC maximization on highly imbalanced data is limited by two-level compositional minimax formulations, which update the backbone only once per round; extending to multi-level compositional minimax in federated settings raises biased-gradient and convergence challenges.

Approach

Formulate AUC maximization as a federated multi-level compositional minimax problem where the backbone takes K inner gradient-descent steps before the AUC loss, and estimate each inner level with STORM-like estimators and momentum-based primal-dual updates, achieving level-independent O(1/sqrt(NT)) convergence.

Key results

• LocalSMCGDAM consistently outperforms all baselines (FedProx, Scaffold, LocalSGDM-RL, CoDA, LocalSGDAM, LocalSCGDAM) on all five imbalanced image datasets across communication periods 4, 8, and 16 with imratio 0.05.
• Theory: convergence rate O(K^2/sqrt(NT)) with linear speedup in number of devices and level-independent order in K; sample complexity O(1/eps^4) and communication complexity O(1/eps^3) per Remark 4.6.
• Too large K (number of levels) degrades AUC; K=3 is used in experiments.

Measured effects reported in the paper

Surprising findings

• Increasing the number of inner levels K beyond a moderate value degrades AUC, attributed to overfitting the imbalanced cross-entropy during pretraining.

Limitations

• Performance degrades for large numbers of levels K, requiring careful choice of K (K=3 used).
• Bias from the multi-level compositional structure makes analysis complex; the constant factor depends exponentially on K as in prior work.
• Experiments limited to image binary classification with N=8 or N=4 simulated workers.

Experimental setup

Datasets: CatvsDog CIFAR10 CIFAR100 STL10 Melanoma

Benchmarks: CatvsDog CIFAR10 CIFAR100 STL10 Melanoma

Metrics: test AUC

Baselines: FedProx Scaffold LocalSGDM-RL CoDA LocalSGDAM LocalSCGDAM

Builds on: LocalSCGDAM CoDA end-to-end deep AUC maximization compositional framework STORM Yang et al. 2019 multi-level compositional optimization

Closest papers by meaning

• Federated Compositional Deep AUC Maximization (NeurIPS 2023 · similarity 0.95)
• LS$^{2}$MC-GDA: A Smoothed Algorithm for Federated Stochastic Multi-Level Compositional Minimax Optimization (ICML 2026 · similarity 0.90)
• Federated Deep AUC Maximization for Hetergeneous Data with a Constant Communication Complexity (ICML 2021 · similarity 0.89)
• A Doubly Recursive Stochastic Compositional Gradient Descent Method for Federated Multi-Level Compositional Optimization (ICML 2024 · similarity 0.88)
• Stochastic AUC Maximization with Deep Neural Networks (ICLR 2020 · similarity 0.88)
• FedNest: Federated Bilevel, Minimax, and Compositional Optimization (ICML 2022 · similarity 0.87)