Research map / Federated learning and privacy / Communication-efficient FL
FedBAT: Communication-Efficient Federated Learning via Learnable Binarization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedBAT, which learns binary model updates and their step size during local federated training via a learnable binarization operator with straight-through gradients, and proves convergence.
Problem
Federated learning incurs heavy uplink communication cost; existing binarization methods (SignSGD variants) binarize model updates post-training, causing approximation errors and requiring hand-tuned step sizes.
Approach
Binarize model updates inside local training (binarization-aware training) with a learnable step size, using a stochastic binarization operator with STE-based gradients for both the updates and the step size, preceded by a full-precision warm-up phase.
Key results
- On FMNIST, SVHN, CIFAR-10, CIFAR-100 with IID/Non-IID partitions and 30/100 clients, FedBAT matches or exceeds FedAvg in several settings and beats binarization baselines by up to 9% accuracy (e.g., CIFAR-100 Non-IID-2 with 100 clients: 49.2 vs FedAvg 49.0).
- FedBAT with rho=6 and warm-up ratio 0.5 achieves 89.0 on FMNIST and 81.0 on CIFAR-10 in ablations, while rho=0 (fixed step size) falls below FedAvg.
- Theory shows FedBAT converges at O(1/T) for strongly convex losses under full participation, matching FedAvg when q=0.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedBAT instead of FedAvg | federated training · FMNIST, SVHN, CIFAR-10 and CIFAR-100; 100 clients, IID and non-IID splits | test accuracy: 91.8 ($\pm$ 0.1) vs 92.0 ($\pm$ 0.1); 90.6 ($\pm$ 0.1) vs 90.5 ($\pm$ 0.2); 89.0 ($\pm$ 0.4) vs 88.7 ($\pm$ 0.2) | Mixed |
| Learn the step size rather than fixing it at its initial value (ablation) | FedBAT · Non-IID-2, 100 clients; FMNIST, SVHN, CIFAR-10 and CIFAR-100 | test accuracy: 87.9 -> 89.0; 88.5 -> 89.2; 78.3 -> 81.0; 41.2 -> 49.2 | Helped |
| Increase ρ from 0 to 2 (ablation) | FedBAT · Non-IID-2, 100 clients; FMNIST, SVHN, CIFAR-10 and CIFAR-100 | test accuracy: 87.9 -> 88.2; 88.5 -> 89.0; 78.3 -> 79.0; 41.2 -> 48.0 | Helped |
| Set ρ within 4, 6, or 8 (ablation) | FedBAT · Non-IID-2, 100 clients; FMNIST, SVHN, CIFAR-10 and CIFAR-100 | test accuracy: FMNIST: 88.9, 89.0, 88.7; SVHN: 89.2, 89.2, 89.5; CIFAR-10: 80.4, 81.0, 80.6; CIFAR-100: 48.7, 49.2, 48.8 | No clear effect |
| Set warm-up ratio φ=0.5 rather than φ=0.1 (ablation) | FedBAT · Non-IID-2, 100 clients; FMNIST, SVHN, CIFAR-10 and CIFAR-100 | test accuracy: 88.6 -> 89.0; 88.4 -> 89.2; 75.9 -> 81.0; 42.9 -> 49.2 | Helped |
| Set warm-up ratio φ=0.5 rather than φ=0.9 (ablation) | FedBAT · Non-IID-2, 100 clients; FMNIST, SVHN, CIFAR-10 and CIFAR-100 | test accuracy: 88.5 -> 89.0; 88.6 -> 89.2; 80.3 -> 81.0; 48.3 -> 49.2 | Helped |
| Use binarization-aware local training instead of post-training binarization | FedBAT vs. binarization baselines · Four datasets, IID and non-IID splits | test accuracy | Helped |
| Use binarization-aware local training | FedBAT · CIFAR-100, 100 clients | convergence speed | Helped |
Surprising findings
- FedBAT sometimes surpasses FedAvg in accuracy despite 1-bit uplink updates (e.g., CIFAR-100 Non-IID-2 with 100 clients).
Limitations
- Clients must store two copies of parameters (initial model and model updates), increasing local memory.
- Downlink is not compressed; the server broadcast remains full precision.
- Convergence analysis assumes a fixed step size rather than the learned one and requires strong convexity or other standard assumptions.
Experimental setup
Datasets: FMNIST SVHN CIFAR-10 CIFAR-100
Benchmarks: FMNIST SVHN CIFAR-10 CIFAR-100
Metrics: test accuracy convergence speed
Baselines: FedAvg SignSGD EF-SignSGD Noisy-SignSGD Stoc-SignSGD ZeroFL FedPAQ BiFL
Builds on: FedAvg SignSGD EF-SignSGD Stoc-SignSGD BinaryConnect
Closest papers by meaning
- FedSMU: Communication-Efficient and Generalization-Enhanced Federated Learning through Symbolic Model Updates (ICML 2025 · similarity 0.87)
- Communication-Efficient Federated Learning with Accelerated Client Gradient (CVPR 2024 · similarity 0.86)
- Low Precision Local Training is Enough for Federated Learning (NeurIPS 2024 · similarity 0.86)
- Why Go Full? Elevating Federated Learning Through Partial Network Updates (NeurIPS 2024 · similarity 0.86)
- Layer-wise Update Aggregation with Recycling for Communication-Efficient Federated Learning (NeurIPS 2025 · similarity 0.86)
- Splitting with Importance-aware Updating for Heterogeneous Federated Learning with Large Language Models (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.