Research map / Federated learning and privacy / Communication-efficient FL
SpaFL: Communication-Efficient Federated Learning With Sparse Models And Low Computational Overhead
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
SpaFL learns structured sparse FL models by training per-filter/neuron thresholds that are the only quantities communicated between clients and server, and derives a differential-privacy-based generalization bound linking density to performance.
Problem
Federated learning on resource-constrained clients suffers high communication and computation overhead; existing sparse FL methods use unstructured pruning or fix/poorly optimize structured sparsity and still communicate parameters or train multiple models.
Approach
Define a trainable threshold per filter/neuron that prunes all its connected weights (structured sparsity); communicate only these thresholds and aggregate them into global thresholds, which are then used to update clients' local parameters based on the sign and magnitude of their change.
Key results
- On FMNIST SpaFL reaches 89.21% accuracy with 0.1856 Gbit communication versus 133.8 Gbit for FedAvg (88.73%), i.e. about 0.14% of FedAvg's communication.
- On CIFAR-10 SpaFL reaches 69.75% accuracy vs 61.33% for FedAvg, using about 0.18% of its communication (0.4537 vs 258.36 Gbit).
- On CIFAR-100 SpaFL reaches 40.80% accuracy vs 35.51% for FedAvg, with 4.6080 Gbit vs 10712 Gbit communication and roughly 15% of FedAvg's FLOPs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add parameter updates using importance extracted from global thresholds (ablation) | SpaFL · FMNIST | accuracy: 88.20$\pm$1.10 -> 89.21$\pm$0.25 | Helped |
| add parameter updates using importance extracted from global thresholds (ablation) | SpaFL · CIFAR-10 | accuracy: 68.63$\pm$1.76 -> 69.75$\pm$2.81 | Helped |
| add parameter updates using importance extracted from global thresholds (ablation) | SpaFL · CIFAR-100 | accuracy: 38.96$\pm$0.80 -> 40.80$\pm$0.54 | Helped |
| train and communicate thresholds rather than leave them at initialization (ablation) | threshold-only training with frozen model parameters · FMNIST | accuracy: 10.22$\pm$0.25 -> 65.52$\pm$5.3 | Helped |
| train and communicate thresholds rather than leave them at initialization (ablation) | threshold-only training with frozen model parameters · CIFAR-10 | accuracy: 10.38$\pm$0.42 -> 60.94$\pm$3.4 | Helped |
| train and communicate thresholds rather than leave them at initialization (ablation) | threshold-only training with frozen model parameters · CIFAR-100 | accuracy: 1.43$\pm$0.53 -> 24.80$\pm$1.1 | Helped |
| use SpaFL instead of FedAvg | federated training · FMNIST | accuracy: 88.73$\pm$0.21 -> 89.21$\pm$0.25 | Mixed |
| use SpaFL instead of FedP3 | federated training · CIFAR-10 | accuracy: 67.54$\pm$0.52 -> 69.75$\pm$2.81 | Helped |
| use SpaFL instead of Fjord | federated training · CIFAR-100 | accuracy: 39.13$\pm$0.22 -> 40.80$\pm$0.54 | Helped |
| use SpaFL instead of FedAvg | federated training · CIFAR-10 with ViT | accuracy: 59.20$\pm$0.4 -> 69.78$\pm$2.62 | Helped |
Surprising findings
- Training only the thresholds with all parameters frozen already yields large accuracy gains over initialization (e.g. 65.52% vs 10.22% on FMNIST), suggesting learned structure alone improves performance.
- Local-only training without threshold communication reaches lower accuracy and higher density than SpaFL, indicating shared thresholds make models sparser and more accurate.
Limitations
- Sparsity cannot be explicitly controlled; the regularizer must be tuned across multiple runs to reach a target density.
- The generalization analysis requires a bounded loss function, which may not hold in practice.
- Convergence analysis is only given for the single-local-epoch case E=1.
Experimental setup
Datasets: FMNIST CIFAR-10 CIFAR-100
Benchmarks: FMNIST CIFAR-10 CIFAR-100
Metrics: accuracy communication cost (Gbit) FLOPs model density
Baselines: FedAvg FedPM HeteroFL Fjord FedSpa FedP3 Local
Builds on: FedAvg FedPM HeteroFL FedSpa
Closest papers by meaning
- SparsyFed: Sparse Adaptive Federated Learning (ICLR 2025 · similarity 0.91)
- An Efficient and Accurate Dynamic Sparse Training Framework Based on Parameter-Freezing (AAAI 2025 · similarity 0.89)
- Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimization (ICLR 2025 · similarity 0.88)
- Achieving Lossless Gradient Sparsification via Mapping to Alternative Space in Federated Learning (ICML 2024 · similarity 0.88)
- Decentralized Sporadic Federated Learning: A Unified Algorithmic Framework with Convergence Guarantees (ICLR 2025 · similarity 0.88)
- FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.