Research map / Federated learning and privacy / Communication-efficient FL
FedMef: Towards Memory-efficient Federated Dynamic Pruning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedMef, a memory-efficient federated dynamic pruning framework combining budget-aware extrusion (transferring information from low-magnitude parameters) and scaled activation pruning (NSConv-based activation caching) to cut accuracy loss and activation memory on resource-constrained devices.
Problem
Federated learning on resource-constrained edge devices needs efficient training, but existing federated dynamic pruning methods suffer post-pruning accuracy degradation from discarding low-magnitude parameters and do not reduce activation memory, which dominates footprint for models like MobileNetV
Approach
Instead of directly discarding low-magnitude parameters, a surrogate L2 penalty on to-be-pruned weights extrudes their information into surviving weights under a budgeted learning-rate schedule; activation caches are pruned during training, made viable without batch normalization by using normalized sparse convolution (NSConv) with filter-wise weight standardization.
Key results
- On CIFAR-10 with MobileNetV2, FedMef improves accuracy by 2.13% over the best baseline (FedTiny) while reducing memory footprint by 28.5%.
- At 90% target sparsity on CIFAR-10 with ResNet18, FedMef reaches 78.1% mean accuracy versus 76.1% for FedTiny and 76.2% for FedDST.
- With batch size 1 on CIFAR-10 ResNet18, FedMef outperforms all baseline federated pruning methods; ablations show both BaE and SAP contribute to accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedMef instead of FedTiny | CIFAR-10, ResNet18 with 90% parameter sparsity · CIFAR-10 | accuracy: 76.1% -> 78.1% | Helped |
| Use FedMef instead of FedTiny | CIFAR-10, ResNet18 with 95% parameter sparsity · CIFAR-10 | accuracy: 71.8% -> 73.6% | Helped |
| Use FedMef instead of FedTiny | CIFAR-10, ResNet18 with 80% parameter sparsity · CIFAR-10 | accuracy: 80.3% -> 81.7% | Helped |
| Add budget-aware extrusion (ablation) | FedMef · CIFAR-10, ResNet18 | accuracy | Helped |
| Add scaled activation pruning with NSConv (ablation) | FedMef · CIFAR-10, ResNet18 | accuracy | Helped |
| Add scaled activation pruning (ablation) | MobileNetV2 · CIFAR-10 | activation memory: 233.1MB -> 70.1MB | Helped |
| Use FedMef instead of FedTiny | MobileNetV2 · CIFAR-10 | memory footprint: 28.5% | Helped |
Limitations
- Excludes memory-intensive frameworks like ZeroFL and PruneFL from comparison
- FedTiny's adaptive batch normalization module was removed from experiments due to memory overhead
- Experiments limited to image classification with ResNet18 and MobileNetV2 on three datasets
Experimental setup
Datasets: CIFAR-10 CINIC-10 TinyImageNet
Benchmarks: CIFAR-10 CINIC-10 TinyImageNet
Metrics: mean accuracy memory footprint ratio training FLOPs data exchange
Baselines: FL-PQSU FedDST FedTiny FedAvg ZeroFL PruneFL
Builds on: DropIT RigL Dual Lottery Ticket Hypothesis REX learning rate schedule FedTiny
Closest papers by meaning
- Device-Wise Federated Network Pruning (CVPR 2024 · similarity 0.90)
- FedRTS: Federated Robust Pruning via Combinatorial Thompson Sampling (NeurIPS 2025 · similarity 0.87)
- SpaFL: Communication-Efficient Federated Learning With Sparse Models And Low Computational Overhead (NeurIPS 2024 · similarity 0.85)
- FedEL: Federated Elastic Learning for Heterogeneous Devices (NeurIPS 2025 · similarity 0.85)
- A High-Efficiency Federated Learning Method Using Complementary Pruning for D2D Communication (Student Abstract) (AAAI 2025 · similarity 0.85)
- FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-Aware Model Update (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.