Research map / Federated learning and privacy / Communication-efficient FL
Layer-wise Update Aggregation with Recycling for Communication-Efficient Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedLUAR, which recycles previous server-side updates for a few layers chosen by a gradient-to-weight ratio metric (stochastically sampled), cutting FL communication to about 17-61% of FedAvg while keeping accuracy.
Problem
Communication cost is a major bottleneck in federated learning; existing communication-efficient methods drop parameter updates, which compromises learning capability.
Approach
Instead of dropping updates for less important layers, the server reuses (recycles) the previous round's aggregated updates for layers selected by the ratio of update norm to weight norm, sampled with probability inversely proportional to that ratio; clients skip uploading those layers.
Key results
- On AG News, FedLUAR matches FedAvg accuracy (82.80% vs 82.66%) at 0.17 communication cost.
- On FEMNIST, FedLUAR achieves 73.17% vs FedAvg 71.01% at 0.18 communication cost.
- On CIFAR-10 (ResNet20), FedLUAR reaches 60.15% vs FedAvg 61.27% at 0.47 communication cost, outperforming the compared communication-efficient baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply layer-wise update recycling | FedAvg on CIFAR-10 · CIFAR-10 (ResNet20) | Comm: 1.00 -> 0.47 | Helped |
| apply layer-wise update recycling | FedAvg on CIFAR-100 · CIFAR-100 (WRN-28) | Comm: 1.00 -> 0.61 | Helped |
| apply layer-wise update recycling | FedAvg on FEMNIST · FEMNIST (CNN) | Comm: 1.00 -> 0.18 | Helped |
| apply layer-wise update recycling | FedAvg on AG News · AG News (DistillBERT) | Comm: 1.00 -> 0.17 | Helped |
| apply layer-wise update recycling | FedAvg on CIFAR-10 · CIFAR-10 (ResNet20) | accuracy: 61.27 -> 60.15 | No clear effect |
| apply layer-wise update recycling | FedAvg on CIFAR-100 · CIFAR-100 (WRN-28) | accuracy: 59.88 -> 59.73 | No clear effect |
| apply layer-wise update recycling | FedAvg on FEMNIST · FEMNIST (CNN) | accuracy: 71.01 -> 73.17 | Helped |
| apply layer-wise update recycling | FedAvg on AG News · AG News (DistillBERT) | accuracy: 82.66 -> 82.80 | No clear effect |
| select layers with the gradient-to-weight ratio rather than uniformly at random (ablation) | LUAR · CIFAR-10, FEMNIST, AG News | accuracy: 53.94\% -> 60.15\%; 71.10\% -> 73.17\%; 80.27\% -> 82.80\% | Helped |
| select layers with the gradient-to-weight ratio rather than gradient norm (ablation) | LUAR · CIFAR-10, FEMNIST, AG News | accuracy: 55.88 \% -> 60.15 \%; 70.91 \% -> 73.17 \%; 75.06\% -> 82.80\% | Helped |
| use stochastic instead of deterministic layer selection (ablation) | LUAR · CIFAR-10, FEMNIST, AG News | accuracy: 48.47\% -> 60.15 \%; 69.08 \% -> 73.17 \%; 80.22\% -> 82.80\% | Helped |
| recycle selected updates instead of dropping them (ablation) | LUAR · CIFAR-10 | accuracy: 46.89 \pm 0.1\% -> 50.07 \pm 1.6\% | Helped |
Surprising findings
- Recycling updates outperforms simply dropping the same layers at the same communication cost (e.g., CIFAR-10 50.07% vs 46.89%).
- Deterministic recycling of the lowest-ratio layers degrades accuracy relative to stochastic selection, and FedLUAR's FEMNIST accuracy exceeds FedAvg in the main table.
Limitations
- Server must inform clients of recycled layer IDs each round, adding small overhead.
- Theoretical guarantee converges only to a neighborhood of a stationary point, with bound growing with recycled layers under non-IID data.
- Experiments use FedAvg as base optimizer in the main results; no LLM fine-tuning experiments.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 FEMNIST AG News
Benchmarks: CIFAR-10 CIFAR-100 FEMNIST AG News
Metrics: validation accuracy communication cost relative to FedAvg memory footprint
Baselines: LBGM FedPAQ FedPara PruneFL FedDropoutAvg FedBAT FedAvg FedProx MOON FedOpt FedMut FedACG
Builds on: FedAvg FedPAQ PruneFL FedDropoutAvg FedBAT
Closest papers by meaning
- Communication-Efficient Federated Learning with Accelerated Client Gradient (CVPR 2024 · similarity 0.89)
- Why Go Full? Elevating Federated Learning Through Partial Network Updates (NeurIPS 2024 · similarity 0.88)
- ConFREE: Conflict-free Client Update Aggregation for Personalized Federated Learning (AAAI 2025 · similarity 0.88)
- The Panaceas for Improving Low-Rank Decomposition in Communication-Efficient Federated Learning (ICML 2025 · similarity 0.88)
- Efficient Adaptive Federated Optimization (NeurIPS 2025 · similarity 0.88)
- FedSMU: Communication-Efficient and Generalization-Enhanced Federated Learning through Symbolic Model Updates (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.