Research map / Federated learning and privacy / Communication-efficient FL
Breaking the $\sqrt{d}$ Communication Barrier in Federated Sampling with Adaptive Hamiltonian Monte Carlo
Jiajun Liang, Linxuan Wang, Guang Lin, Qifan Song
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Adaptive Federated Hamiltonian Monte Carlo (AFHMC) for Bayesian federated learning, aiming to beat the O(√d/ε) communication cost of prior federated HMC sampling.
Problem
Communication cost of sampling from posteriors in Bayesian federated learning; prior federated HMC achieves O(√d/ε) communication under log-convex distributions.
Approach
Adaptive federated HMC with a refined analysis that reveals further benefits of HMC in the FL regime, aiming to break the √d communication barrier.
Key results
- Prior federated HMC achieves O(√d/ε) communication cost under log-convex distributions; the paper claims a refined analysis showing improvement beyond this barrier (exact bound not stated in the abstract provided).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add global drift control variate and surrogate leapfrog to federated HMC | AFHMC vs FA-HMC · Bayesian logistic regression d=1000, eta=5e-4 | marginal error: 5.3e-2 -> 2.3e-2 | Improved |
| add drift control variate (AFHMC) under matched leapfrog budget | federated HMC · Heterogeneous Gaussian d=500 | squared W2 distance: 3.81e4 (FA-HMC) vs below 0.1 threshold reached by AFHMC at round 430 | Improved |
| give FA-HMC twice the rounds and its own tuned stepsize schedules (ablation) | FA-HMC · Heterogeneous Gaussian d=500 | squared W2 distance: 1.19e4 and 1.98e4 after 860 rounds | No clear effect |
| lower client heterogeneity (clients sampled from same distribution) (ablation) | AFHMC · Gaussian, d from 50 to 500 | communication rounds: 212 -> 248 vs 162 -> 430 on heterogeneous target | Improved |
| use AFHMC instead of FA-HMC on heterogeneous FMNIST | Bayesian logistic regression · FMNIST (Dirichlet 0.1) | test accuracy: 0.841 -> 0.842 (same exchanges); 0.836 at same rounds | Improved |
Limitations
- Only abstract available; full text unavailable for verification of results.
Experimental setup
Metrics: communication cost
Baselines: federated Langevin dynamics methods prior federated HMC (O(√d/ε))
Builds on: federated Hamiltonian Monte Carlo
Closest papers by meaning
- SCAFFLSA: Taming Heterogeneity in Federated Linear Stochastic Approximation and TD Learning (NeurIPS 2024 · similarity 0.87)
- The Sample-Communication Complexity Trade-off in Federated Q-Learning (NeurIPS 2024 · similarity 0.86)
- Streaming Federated Learning with Markovian Data (NeurIPS 2025 · similarity 0.86)
- Efficient Adaptive Federated Optimization (NeurIPS 2025 · similarity 0.85)
- Federated Q-Learning: Linear Regret Speedup with Low Communication Cost (ICLR 2024 · similarity 0.85)
- Improving Generalization in Federated Learning with Model-Data Mutual Information Regularization: A Posterior Inference Approach (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
