Research map / Federated learning and privacy / Personalized and heterogeneous FL
Data-Free Black-Box Federated Learning via Zeroth-Order Gradient Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedZGE, a data-free black-box federated learning framework that trains a server-side generator using zeroth-order gradient estimation through client model outputs only, avoiding auxiliary data and model parameter sharing.
Problem
Distillation-based FL needs a task-relevant auxiliary dataset, while data-free FL requires clients to share on-device models for white-box generator training, compromising communication, privacy, and black-box advantages.
Approach
Estimate gradients of the generator's loss with respect to synthetic inputs via finite differences using only forward outputs from black-box client models, and train the generator with fidelity, adversarial transferability, diversity, and information-entropy balance losses, plus local distillation of ensemble outputs.
Key results
- On CIFAR-10 with ResNet-50 (alpha=0.1), FedZGE reaches 72.94% vs. 72.72% for DFRD while using 63.17 GB communication vs. 175.30 GB.
- Under the heterogeneous setting on CIFAR-100 (alpha=1), FedZGE achieves 59.73% vs. 59.52% for DFRD with 65.18 GB vs. 160.07 GB.
- FedZGE with q=10 perturbation directions reaches accuracy close to true-gradient FedZGE; accuracy grows with q at the cost of communication.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedZGE instead of DFRD | ResNet-50 heterogeneous FL · CIFAR-10, α=1 | accuracy: 83.21$\pm$0.22 -> 83.60$\pm$0.14 | Helped |
| Use FedZGE instead of DFRD | ResNet-50 homogeneous FL · CIFAR-100, α=0.1 | accuracy: 44.51$\pm$0.33 -> 44.67$\pm$0.36 | Helped |
| Set perturbation directions q=1 instead of q=10 (ablation) | FedZGE · CIFAR-10, α=0.1, ResNet-50 | accuracy: 72.94$\pm$0.25 -> 72.08$\pm$0.22 | Hurt |
| Set perturbation directions q=5 instead of q=10 (ablation) | FedZGE · CIFAR-10, α=0.1, ResNet-50 | accuracy: 72.94$\pm$0.25 -> 72.33$\pm$0.33 | Hurt |
| Set perturbation directions q=20 instead of q=10 (ablation) | FedZGE · CIFAR-10, α=0.1, ResNet-50 | accuracy: 72.94$\pm$0.25 -> 73.00$\pm$0.40 | Helped |
| Use only the fidelity loss (ablation) | FedZGE generator · CIFAR-10, α=0.1, ResNet-50 | accuracy: 72.94$\pm$0.25 -> 72.01$\pm$0.13 | Hurt |
| Add the adversarial loss (ablation) | FedZGE generator · CIFAR-10, α=0.1, ResNet-50 | accuracy: 72.70$\pm$0.37 -> 72.94$\pm$0.25 | Helped |
| Add the diversity loss (ablation) | FedZGE generator · CIFAR-10, α=0.1, ResNet-50 | accuracy: 72.82$\pm$0.21 -> 72.94$\pm$0.25 | Helped |
| Add the information entropy loss (ablation) | FedZGE generator · CIFAR-100, α=0.1, ResNet-50 | accuracy: 44.44$\pm$0.25 -> 44.44$\pm$0.30 | No clear effect |
| Add client local distillation loss (ablation) | FedZGE · CIFAR-100, α=0.1, ResNet-50 | accuracy: 44.30$\pm$0.33 -> 44.67$\pm$0.36 | Helped |
Surprising findings
- FedZGE slightly outperforms data-free white-box methods such as DFRD and FedFTG in several settings despite using only black-box outputs.
- The gap to true-gradient training widens as local model architectures become more complex.
Limitations
- Advantage diminishes as the number of clients grows because gradient estimation error accumulates across more local models.
- Communication overhead is higher than distillation-based FL due to q perturbation queries per round.
- Evaluated only on image classification with small-scale CIFAR datasets; textual domains and large foundation models are not explored.
Experimental setup
Datasets: CIFAR-10 CIFAR-100
Benchmarks: CIFAR-10 CIFAR-100
Metrics: test accuracy communication overhead (GB)
Baselines: FedAvg MHAT DS-FL FedGen FedFTG DFRD FedZKT
Builds on: FedAvg data-free knowledge distillation FedZKT DFRD FedFTG DS-FL
Closest papers by meaning
- Federated Learning via Input-Output Collaborative Distillation (AAAI 2024 · similarity 0.91)
- Federated Black-Box Adaptation for Semantic Segmentation (NeurIPS 2024 · similarity 0.88)
- FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning (NeurIPS 2025 · similarity 0.88)
- FLUX: Efficient Descriptor-Driven Clustered Federated Learning under Arbitrary Distribution Shifts (NeurIPS 2025 · similarity 0.88)
- Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimization (ICLR 2025 · similarity 0.88)
- Efficient Federated Learning via Clients-to-Server Knowledge Distillation (Student Abstract) (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.