Research map / Federated learning and privacy / Communication-efficient FL
Converge Faster, Talk Less: Hessian-Informed Federated Zeroth-Order Optimization
Zhe Li, Bicheng Ying, Zidong Liu, Chaosheng Dong, Haibo Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Hessian-informed preconditioning (HiSo vs DeComFL) | DeComFL · SQuAD (OPT-350M) | rounds to match accuracy: 1350 -> 250 rounds (5.4x) | Improved |
| add Hessian-informed preconditioning | DeComFL · SST-2 (OPT-125M) | test accuracy: 85.21 -> 85.55 | Improved |
| add Hessian-informed preconditioning | DeComFL · QQP (OPT-350M) | test accuracy: 60.58 -> 62.49 | Improved |
| add Hessian-informed preconditioning | DeComFL · SST-2 (OPT-1.3B) | test accuracy: 90.22 -> 90.34 | No clear effect |
| add Hessian-informed preconditioning | DeComFL · QQP (OPT-1.3B) | communication cost (KB): 43.95 -> 96.67 | Hurt |
| use HiSo instead of DeComFL on newer LLMs | DeComFL · SST-2 (Qwen3-0.6B) | communication rounds: 2500 -> 1000 rounds | Improved |
| use HiSo instead of DeComFL | DeComFL · SST-2 (Qwen3-0.6B wall-clock) | total wall-clock time (s): 2350 -> 1064 | Improved |
| add LoRA (r=8) to HiSo (ablation) | HiSo · SST-2 (OPT-125M) | test accuracy: 85.55 -> 85.37 | Hurt |
| vary Hessian smoothing parameter nu (ablation) | HiSo · MNIST CNN | convergence and final accuracy | No clear effect |
| HiSo vs first-order FedAdam | HiSo · SST-2 (OPT-1.3B) | test accuracy: 92.86 -> 90.34 | Hurt |
Closest papers by meaning
- Zeroth-Order Methods for Nondifferentiable, Nonconvex, and Hierarchical Federated Optimization (NeurIPS 2023 · similarity 0.90)
- Countering the Communication Bottleneck in Federated Learning: A Highly Efficient Zero-Order Optimization Technique (JMLR 2024 · similarity 0.90)
- Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimization (ICLR 2025 · similarity 0.90)
- Federated Optimization with Doubly Regularized Drift Correction (ICML 2024 · similarity 0.88)
- SILVER: Single-loop variance reduction and application to federated learning (ICML 2024 · similarity 0.88)
- Zeroth-Order Optimization Finds Flat Minima (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
