Research map / Federated learning and privacy / Communication-efficient FL
Device-Wise Federated Network Pruning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DWNP, a federated channel-pruning method that generates a server-side sub-network and per-device sub-networks from embeddings via a hypernetwork, trained jointly with FL and finetuned, with a convergence guarantee.
Problem
Channel pruning of CNNs for federated learning under non-iid device data, where data-driven pruning cannot be applied directly because data are distributed and confidential.
Approach
Generates device-wise sub-networks from learned network embeddings through a shared hypernetwork (Gumbel-sigmoid channel gates), permanently prunes a server-side sub-network that serves as the weight bank for device sub-networks, and alternates weight and hypernetwork training, framed as federated bilevel optimization.
Key results
- On CIFAR-100 with ResNet-34, DWNP reaches +2.17% over the base model while pruning 50% device-side FLOPs (20% server-side).
- On CIFAR-10 with ResNet-56, DWNP improves accuracy by +0.66% (91.88%) at 50% device FLOPs, versus -0.93% for Filter Pruning.
- On TinyImageNet with ResNet-18, DWNP gains +1.06% Top-1 and +1.10% Top-5 over the base model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use device-wise sub-networks with a server-side sub-network instead of a shared pruning configuration | DWNP · CIFAR-100, ResNet-18 | accuracy: 66.57% -> 68.31% | Helped |
| use device-wise sub-networks with a server-side sub-network instead of a shared pruning configuration | DWNP · CIFAR-10, ResNet-56 | accuracy: 91.22% -> 91.88% | Helped |
| use device-wise sub-networks with a server-side sub-network instead of a shared pruning configuration | DWNP · CIFAR-100, ResNet-34 | accuracy: 69.05% -> 71.72% | Helped |
| increase the number of devices from N=10 to N=50 (ablation) | DWNP · CIFAR-100, ResNet-18 | accuracy: 68.31% -> 67.58% | No clear effect |
| decrease Dirichlet α from 0.5 to 0.1 (ablation) | DWNP · CIFAR-100, ResNet-18 | accuracy: 68.31% -> 64.51% | Hurt |
| decrease Dirichlet α from 0.5 to 0.05 (ablation) | DWNP · CIFAR-100, ResNet-18 | accuracy: 68.31% -> 62.59% | Hurt |
| decrease Dirichlet α from 0.5 to 0.1 (ablation) | DWNP · CIFAR-100, ResNet-34 | accuracy: 71.72% -> 68.20% | Hurt |
| increase hypernetwork update interval rHN to 30 (ablation) | DWNP · CIFAR-100, ResNet-18 | test accuracy | Hurt |
| increase hypernetwork update interval rHN to 10 or more (ablation) | DWNP · CIFAR-100, ResNet-18 | communication cost | Helped |
| learn model weights and pruning architectures simultaneously rather than learning the architecture in one shot | FedILP versus FedOSP · CIFAR-100 | accuracy | Helped |
Surprising findings
- Pruned device-side sub-networks exceed the unpruned base model accuracy on CIFAR-100 and TinyImageNet, even at 70% FLOPs pruning on ResNet-18.
Limitations
- Server-side sub-network achieves much smaller FLOPs reduction (20%) and requires more server storage than baselines.
- Assumes training and test data distributions are similar within each device.
- Convergence analysis relies on mild assumptions and ignores the indirect gradient term in practice.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 TinyImageNet
Benchmarks: CIFAR-10 CIFAR-100 TinyImageNet
Metrics: top-1 accuracy top-5 accuracy accuracy change vs base model FLOPs reduction
Baselines: Filter Pruning FedOSP FedILP
Builds on: FedAvg Filter Pruning (Li et al.) Straight-through Gumbel-Sigmoid Mime
Closest papers by meaning
- FedMef: Towards Memory-efficient Federated Dynamic Pruning (CVPR 2024 · similarity 0.90)
- Pruning from Scratch (AAAI 2020 · similarity 0.89)
- GDeR: Safeguarding Efficiency, Balancing, and Robustness via Prototypical Graph Pruning (NeurIPS 2024 · similarity 0.87)
- A High-Efficiency Federated Learning Method Using Complementary Pruning for D2D Communication (Student Abstract) (AAAI 2025 · similarity 0.87)
- FedRTS: Federated Robust Pruning via Combinatorial Thompson Sampling (NeurIPS 2025 · similarity 0.86)
- How Sparse Can We Prune A Deep Network: A Fundamental Limit Perspective (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.