Research map / Federated learning and privacy / Communication-efficient FL
Why Go Full? Elevating Federated Learning Through Partial Network Updates
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedPart, which in federated learning trains and averages only one or a few layers per round (sequential, multi-round cyclic layer selection) to mitigate layer mismatch, reducing communication and computation while improving accuracy.
Problem
Full-network parameter updates and averaging in federated learning cause "layer mismatch" (layers of the averaged global model fail to cooperate), slowing convergence and lowering final accuracy compared with centralized training.
Approach
Restrict each communication round to updating a single layer or a few layers, selected sequentially from shallow to deep and repeated in multiple cycles (inspired by block coordinate descent), so frozen layers anchor the trainable layer and the averaged layers align better.
Key results
- On Tiny-ImageNet with ResNet-8, FedPart improves FedAvg accuracy by about 21% while using 28% of FNU communication and 73% of computation.
- With ResNet-18 on Tiny-ImageNet, FedPart reduces communication by 85% and computation by 27% relative to full network updates while achieving higher accuracy (17.1% vs 13.7%).
- Theorem 1 gives a convergence rate of O(1/sqrt(MNT)) for FedPart versus O(1/sqrt(NT)) for full network updates under non-convex L-smooth assumptions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use partial network updates instead of full-network updates | FedAvg on ResNet-18 · Tiny-ImageNet | Best Acc.: 13.7 (±0.2) -> 17.1 (±0.2) | Helped |
| use partial network updates instead of full-network updates | FedAvg on ResNet-18 · CIFAR-10 | Best Acc.: 61.7 (±0.2) -> 59.2 (±0.4) | Hurt |
| use partial network updates instead of full-network updates | FedAvg on ResNet-18 · CIFAR-100 | Best Acc.: 32.0 (±0.5) -> 33.4 (±0.4) | Helped |
| use partial network updates instead of full-network updates | FedAvg on a Transformer · AG News | Best Acc.: 92.1 (±0.3) -> 92.0 (±0.3) | No clear effect |
| use partial network updates instead of full-network updates | FedAvg on a Transformer · Sogou News | Best Acc.: 94.4 (±0.2) -> 94.4 (±0.2) | No clear effect |
| remove initial full-network warm-up (ablation) | FedPart · CIFAR-100 | accuracy after partial updates: 33.59 -> 29.53 | Hurt |
| extend initial full-network warm-up from 5 to 60 rounds (ablation) | FedPart · CIFAR-100 | accuracy after partial updates: 33.59 -> 36.65 | Helped |
| select layers in reverse rather than sequential order (ablation) | FedPart · CIFAR-100 | accuracy: 33.59 -> 31.79 | Hurt |
| select layers randomly rather than sequentially (ablation) | FedPart · CIFAR-100 | accuracy: 33.59 -> 31.44 | Hurt |
| increase training rounds per layer from R/L=1 to R/L=10 (ablation) | FedPart · CIFAR-100 | accuracy: 28.10 -> 22.83 | Hurt |
| increase r from 15 to 45 with R/L=2 (ablation) | FedPart · CIFAR-100 | accuracy: 24.47 -> 32.53 | Helped |
| transmit only a subset of parameters rather than all parameters | FedPart (5C) · image reconstruction privacy attack | Avg. PSNR: 17.07 -> 12.53 | Helped |
Surprising findings
- On CIFAR-10 the accuracy gains are small, which the authors attribute to client drift dominating on simpler datasets rather than layer mismatch.
- With extreme data heterogeneity (Dirichlet alpha=0.1), FedPart is roughly on par with full-network accuracy, with its benefit mainly in reduced communication and computation.
Limitations
- Evaluated only on small models (ResNet-8, ResNet-18) and small image/text datasets; scalability to larger models and datasets is untested.
- Gains are smaller under strong data heterogeneity and on simpler datasets such as CIFAR-10, where client drift dominates.
- Convergence analysis relies on an approximate mask-variance assumption (Assumption 3) justified only by Monte Carlo estimates of k.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny-ImageNet AG News Sogou News
Benchmarks: CIFAR-10 CIFAR-100 Tiny-ImageNet AG News Sogou News
Metrics: best accuracy communication cost (GB) computation cost (TFLOPs) SSIM PSNR
Baselines: FedAvg FedProx FedMOON Full network updates (FNU) FedAvg-FNU
Builds on: FedAvg FedProx MOON Block Coordinate Descent
Closest papers by meaning
- Layer-wise Update Aggregation with Recycling for Communication-Efficient Federated Learning (NeurIPS 2025 · similarity 0.88)
- FedImpro: Measuring and Improving Client Update in Federated Learning (ICLR 2024 · similarity 0.87)
- Splitting with Importance-aware Updating for Heterogeneous Federated Learning with Large Language Models (ICML 2025 · similarity 0.87)
- FedLMT: Tackling System Heterogeneity of Federated Learning via Low-Rank Model Training with Theoretical Guarantees (ICML 2024 · similarity 0.87)
- Communication-Efficient Federated Learning with Accelerated Client Gradient (CVPR 2024 · similarity 0.87)
- FedBAT: Communication-Efficient Federated Learning via Learnable Binarization (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.