Research map / Federated learning and privacy / Communication-efficient FL
$\texttt{pfl-research}$: simulation framework for accelerating research in Private Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces pfl-research, a fast modular Python framework for simulating (private) federated learning that simulates computation without topology, and benchmarks it against other FL simulators for speed and provides a PFL benchmark suite.
Problem
Existing open-source federated learning simulators are too slow to run experiments on large, realistic FL datasets, especially with differential privacy, limiting research productivity.
Approach
Simulate only the computation of FL (no coordinator/topology), keep one model per worker process resident on GPU with in-place updates, load-balance users across workers, and run DP mechanisms on GPU as composable postprocessors, allowing multiple processes to share a GPU.
Key results
- pfl-research PyTorch with 5 processes sharing one GPU trains CIFAR10 in 4.20 minutes, 7x faster than Flower (10 processes) and 72x faster than FedScale.
- On FLAIR, pfl-research 0.2.0 takes 1.77 hours versus 18 hours for TensorFlow Federated (10.2x) and 30.5 hours for Flower (17.2x).
- Banded matrix factorization (DP-FTRL) gives about 10% relative perplexity improvement over the Gaussian mechanism with PLD accounting on StackOverflow; SCAFFOLD does not beat FedAvg on any benchmark.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use pfl-research (PyTorch) with p=5 instead of p=1 | pfl-research · CIFAR10 IID | wall-clock time: $10.13_{\pm 0.06}$ -> $4.20_{\pm 0.10}$ | Helped |
| use pfl-research instead of alternative open-source simulators | CIFAR10 IID simulation · CIFAR10 IID | wall-clock time: 7-72$\times$ faster | Helped |
| upgrade from pfl-research 0.1.0 to 0.2.0 | pfl-research · FLAIR | wall-clock time: 2.16 -> 1.77 | Helped |
| enable central DP (ablation) | pfl-research 0.2.0 · FLAIR | wall-clock time: 1.77 -> 1.93; $9\%$ extra simulation wall-clock time | Hurt |
| use banded matrix factorization instead of Gaussian mechanism with PLD accounting | FedAvg with central DP · StackOverflow | perplexity: $69.11_{\pm .35}$ -> $62.47_{\pm .22}$; 10\% relative improvement | Helped |
| use FedProx instead of FedAvg | federated training · FLAIR IID | mAP: $65.08_{\pm .60}$ -> $64.45_{\pm 1.13}$ | Hurt |
| use FedProx instead of FedAvg | federated training · FLAIR | mAP: $61.83_{\pm .10}$ -> $62.02_{\pm .11}$ | Helped |
| use SCAFFOLD instead of FedAvg | federated training · CIFAR10, StackOverflow, and FLAIR benchmarks | benchmark performance | Hurt |
| increase from 1 to 5 processes per GPU | pfl-research distributed simulations · CIFAR10 and StackOverflow | wall-clock time: less than $0.43$ of that with $p=1$ | Helped |
| add greedy scheduling of users (ablation) | pfl-research worker scheduling · FLAIR | maximum straggler time: $1294$ -> $484$ | Helped |
| add median datapoint count as a base to greedy scheduling weights (ablation) | pfl-research worker scheduling · FLAIR | maximum straggler time: $484$ -> $178$ | Helped |
| add a base value close to the median number of datapoints per user to scheduling weights (ablation) | pfl-research worker scheduling · FLAIR | wall-clock time: additional $3\%$ speedup; total $19\%$ speedup compared to no worker scheduling | Helped |
Surprising findings
- SCAFFOLD does not outperform FedAvg on any of the benchmarks.
- Adding DP to FLAIR adds only about 9% wall-clock overhead in simulation.
Limitations
- Benchmarks are only for PyTorch; TensorFlow benchmark results left for future work.
- Only cross-device benchmarks; no cross-silo benchmarks.
- Not intended for deployment on real edge devices; no model parallelism.
Experimental setup
Datasets: CIFAR10 StackOverflow FLAIR Stanford Alpaca Aya OpenAssistant
Benchmarks: CIFAR10 StackOverflow FLAIR Stanford Alpaca Aya OpenAssistant
Metrics: wall-clock time accuracy perplexity mAP
Baselines: FedML TensorFlow Federated Flower FedScale FLUTE
Builds on: FedAvg FedAdam DP-FTRL Horovod
Closest papers by meaning
- FedLMT: Tackling System Heterogeneity of Federated Learning via Low-Rank Model Training with Theoretical Guarantees (ICML 2024 · similarity 0.89)
- FLUX: Efficient Descriptor-Driven Clustered Federated Learning under Arbitrary Distribution Shifts (NeurIPS 2025 · similarity 0.87)
- Federated Learning with Heterogeneous LLMs: Integrating Small Student Client Models with a Large Hungry Model (AAAI 2025 · similarity 0.87)
- FLHetBench: Benchmarking Device and State Heterogeneity in Federated Learning (CVPR 2024 · similarity 0.87)
- A Bayesian Approach for Personalized Federated Learning in Heterogeneous Settings (NeurIPS 2024 · similarity 0.87)
- SparsyFed: Sparse Adaptive Federated Learning (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.