Research map / Federated learning and privacy / Communication-efficient FL
FedRTS: Federated Robust Pruning via Combinatorial Thompson Sampling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedRTS, a federated dynamic pruning framework that adjusts sparse topologies via combinatorial Thompson sampling (TSAdj) over Beta posteriors per weight, reducing communication by uploading only top-gradient indices.
Problem
Federated pruning with dynamic sparse training suffers from greedy, unstable topology adjustments under data heterogeneity and partial client participation, plus high communication cost from uploading full gradients.
Approach
Reframe topology adjustment as a combinatorial multi-armed bandit and replace deterministic magnitude-based pruning/regrowth with Thompson sampling over per-weight Beta posteriors updated from fused individual and aggregated outcomes; clients upload only top-K-kappa gradient indices.
Key results
- On CIFAR-10 with ResNet18, FedRTS achieves a 5.1% accuracy improvement over the best baseline at similar communication cost, per the paper.
- FedRTS reports a 33.3% communication cost reduction at matched accuracy on CIFAR-10 with ResNet18 (introduction claim).
- On TinyStories with GPT-2-32M, FedRTS achieves lower perplexity than baselines and even surpasses full-size FedAvg at low densities.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedRTS instead of the best baseline | ResNet18 · CIFAR-10 | accuracy: 5.1% | Helped |
| Use FedRTS instead of a SOTA framework | ResNet18 · CIFAR-10 | communication cost: 33.3% reduction | Helped |
| Use FedRTS instead of federated pruning baselines | GPT-2-32M · TinyStories | perplexity | Helped |
| Use FedRTS instead of baselines as data heterogeneity increases | FedRTS · CIFAR-10 and TinyStories, varying data heterogeneity | accuracy | Helped |
| Use FedRTS instead of baselines at lower client participation | FedRTS · CIFAR-10, varying client participation | accuracy | Helped |
| Exclude aggregated information from probability updates (ablation) | FedRTS · CIFAR-10, various target densities | accuracy | No clear effect |
| Add individual information to aggregated information through fusion (ablation) | FedRTS (γ = 1) · CIFAR-10, various target densities | accuracy | Helped |
| Use probabilistic rather than deterministic topology adjustment (ablation) | FedRTS without TS (γ = 1) · CIFAR-10, various target densities | accuracy | Helped |
| Add farsighted historical information (ablation) | FedTiny · CIFAR-10, various target densities | accuracy | Helped |
Surprising findings
- FedRTS reaches lower perplexity than full-size FedAvg at low density levels on TinyStories, which the authors attribute to probabilistic adjustment.
Limitations
- Theory relies on strong assumptions (independent importance scores, mean-field approximation) that hold only approximately for weight magnitudes.
- Evaluation limited to lightweight models (ResNet18, ShuffleNetV2, GPT-2-32M) and 100-client cross-device setup.
Experimental setup
Datasets: CIFAR-10 CINIC-10 TinyImageNet SVHN TinyStories
Benchmarks: CIFAR-10 CINIC-10 TinyImageNet SVHN TinyStories
Metrics: accuracy perplexity communication cost ratio training FLOPs
Baselines: PruneFL FedTiny FedDST FedSGC FedMef FedAVG
Builds on: PruneFL FedTiny FedDST FedMef combinatorial Thompson sampling
Closest papers by meaning
- FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning (NeurIPS 2025 · similarity 0.89)
- SparsyFed: Sparse Adaptive Federated Learning (ICLR 2025 · similarity 0.88)
- SpaFL: Communication-Efficient Federated Learning With Sparse Models And Low Computational Overhead (NeurIPS 2024 · similarity 0.88)
- Beyond Local Sharpness: Communication-Efficient Global Sharpness-aware Minimization for Federated Learning (CVPR 2025 · similarity 0.87)
- An Efficient and Accurate Dynamic Sparse Training Framework Based on Parameter-Freezing (AAAI 2025 · similarity 0.87)
- Byzantine-Robust Federated Learning: Impact of Client Subsampling and Local Updates (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.