Research map / Federated learning and privacy / Personalized and heterogeneous FL
How Does the Smoothness Approximation Method Facilitate Generalization for Federated Adversarial Learning?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Provides on-average algorithmic stability generalization bounds for two federated adversarial learning algorithms (VFAL and SFAL) under three smoothness approximations (SSA, RSA, OPSA), and recommends RSA and SFAL-style reweighting.
Problem
Generalization of federated adversarial learning is not understood because adversarial loss is non-smooth and client data heterogeneity is amplified by adversarial example generation.
Approach
Derives on-average stability generalization bounds for VFAL and SFAL under surrogate, randomized, and over-parameterized smoothness approximations, showing how heterogeneity (total variation between clean and adversarial distributions) and approximation parameters enter the bound, and that slack reweighting aggregation (SFAL) improves the bound via factor r_alpha.
Key results
- Theoretical bounds show RSA gives the best generalization order among SSA, RSA, OPSA (RSA's dominant term T^{3/4}, versus T^{1} for SSA and T^{2} for OPSA as stated in the paper's remark).
- Experiments on SVHN with m=40 clients show the generalization gap grows with attack strength rho and label skew a, and shrinks as the number of clients m increases.
- SFAL's aggregation factor r_alpha reduces the Delta and D_max terms relative to VFAL, so under suitable alpha it reduces generalization error.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RSA instead of SSA or OPSA | VFAL · FAL generalization | generalization error | Helped |
| Use RSA instead of SSA or OPSA | FAL · SVHN and CIFAR10 | generalization gap | Helped |
| Use α-slack loss-based reweighting instead of simple averaging | VFAL · FAL generalization | generalization error | Helped |
| Use α-slack loss-based reweighting instead of simple averaging | VFAL · SVHN and CIFAR10 | generalization gap | Helped |
| Increase attack strength ρ | FAL · SVHN and CIFAR10 | generalization gap | Hurt |
| Increase label-skew parameter a | FAL · SVHN and CIFAR10 | generalization gap | Hurt |
| Increase the number of clients m | FAL · SVHN and CIFAR10 | generalization gap | Helped |
| Increase the number of smoothing-noise samples Q (ablation) | RSA · SVHN | generalization gap | Helped |
| Reduce attack strength ρ (ablation) | SSA · FAL generalization | generalization error: ρ T\log T | Helped |
| Control network width s (ablation) | OPSA · FAL generalization | generalization error: \mathcal{O}(\rho^2\sqrt{s}) | Helped |
| Increase α in α-slack aggregation (ablation) | SFAL · FAL generalization | generalization error: r_\alpha = 1+ \frac{\alpha}{1-\alpha}\frac{2\hat{m}}{m} | Helped |
| Use a larger Q (ablation) | RSA · FAL generalization | generalization error upper bound: \frac{T^{\frac{1}{4}}\log T}{\sqrt{Q}} | Helped |
Surprising findings
- OPSA has the largest generalization error because of a sqrt(s) width term, which the paper calls a curse of width.
Limitations
- Experiments are small-scale (SmallCNN on SVHN and CIFAR10) and check only the trends of the bounds rather than tight constants.
- Bounds rely on smoothness and boundedness assumptions, and the non-smooth approximations introduce approximation error terms.
Experimental setup
Datasets: SVHN CIFAR10
Benchmarks: SVHN CIFAR10
Metrics: generalization gap (train minus test robust accuracy)
Baselines: VFAL SFAL
Builds on: Federated Adversarial Learning FedAvg PGD adversarial training randomized smoothing on-average stability
Closest papers by meaning
- Byzantine-Robust Federated Learning: Impact of Client Subsampling and Local Updates (ICML 2024 · similarity 0.86)
- Efficient Federated Learning against Byzantine Attacks and Data Heterogeneity via Aggregating Normalized Gradients (NeurIPS 2025 · similarity 0.86)
- Lessons from Generalization Error Analysis of Federated Learning: You May Communicate Less Often! (ICML 2024 · similarity 0.85)
- Sharpness-Aware Model-Agnostic Long-Tailed Domain Generalization (AAAI 2024 · similarity 0.85)
- Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimization (ICML 2024 · similarity 0.85)
- FedAA: A Reinforcement Learning Perspective on Adaptive Aggregation for Fair and Robust Federated Learning (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.