Research map / Federated learning and privacy / Communication-efficient FL

An Efficient and Accurate Dynamic Sparse Training Framework Based on Parameter-Freezing

AAAI 2025 · Federated learning and privacy › Communication-efficient FL · approach: Efficient local training in FL · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PFFDST, a federated dynamic sparse training framework combining server-side mask readjustment with differential sparsity and a two-step parameter-freezing scheme to cut communication and FLOPs while improving accuracy over FedDST.

Problem

Federated learning requires frequent parameter synchronization, which is communication-heavy; existing sparse federated methods (e.g., FedDST) suffer accuracy loss and mask mismatches under non-IID data and high sparsity.

Approach

Readjust the sparse mask on the server (rather than on clients) with a differential sparsity that allows a larger readjustment rate, and then freeze already-trained non-zero parameters and mask in a second phase so only the newly grown sub-network is communicated and trained.

Key results

• On ResNet-18, PFFDST improves accuracy on average by 4% on CIFAR-10 and 6% on CIFAR-100 over FedDST under the same communication cost and FLOPs.
• When targeting the same accuracy, PFFDST reduces communication cost by 4-8x across datasets and sparsity levels.
• Parameter-freezing reduces per-round communication by about one-half and training FLOPs by one-third relative to sparse training without freezing.

Measured effects reported in the paper

Surprising findings

• Server-side mask readjustment dominates at low communication budgets, but parameter-freezing gives the larger gain at 4GB; the ranking reverses at 24GB.

Limitations

• Mask readjustment on the server adds communication and FLOPs in readjustment rounds (differential sparsity increases cost by alpha, mitigated by Rend = R/4).
• Evaluated only on small image models (LeNet, ResNet-18) and CIFAR datasets in the main text; LLM extension is only proposed as future work.

Experimental setup

Datasets: CIFAR-10 CIFAR-100

Benchmarks: CIFAR-10 CIFAR-100

Metrics: test accuracy communication cost (GB) training FLOPs

Baselines: FedAvgM FedProx FedDST FedDST+FedProx FedTiny FedMef FLASH

Builds on: FedDST Lottery Ticket Hypothesis FedAvg FedProx

Closest papers by meaning

• SparsyFed: Sparse Adaptive Federated Learning (ICLR 2025 · similarity 0.91)
• SpaFL: Communication-Efficient Federated Learning With Sparse Models And Low Computational Overhead (NeurIPS 2024 · similarity 0.89)
• Low Precision Local Training is Enough for Federated Learning (NeurIPS 2024 · similarity 0.89)
• FedRTS: Federated Robust Pruning via Combinatorial Thompson Sampling (NeurIPS 2025 · similarity 0.87)
• Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimization (ICLR 2025 · similarity 0.87)
• EFSkip: A New Error Feedback with Linear Speedup for Compressed Federated Learning with Arbitrary Data Heterogeneity (AAAI 2025 · similarity 0.87)