Research map / Federated learning and privacy / Personalized and heterogeneous FL

Reviving Stale Updates: Data-Free Knowledge Distillation for Asynchronous Federated Learning

NeurIPS 2026 · Federated learning and privacy › Personalized and heterogeneous FL · approach: Non-IID federated learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FedRevive, an asynchronous FL framework that revives stale client updates via server-side data-free multi-teacher distillation with a meta-learned generator, hybridized with parameter-space aggregation.

Problem

Asynchronous federated learning suffers from update staleness, where delayed client updates computed on outdated global models destabilize optimization, especially under non-IID data.

Approach

Treat buffered stale client models as teachers and the current server model as student; synthesize pseudo-samples with a meta-learned (Reptile-updated) generator and distill multi-teacher knowledge server-side, then mix the resulting KD update with the regular update using a staleness-dependent weight beta(tau).

Key results

• On CIFAR-10, FedRevive reaches 80.51% final accuracy vs 63.99% for FedBuff, with time-to-target 65.5 vs 96.4 simulated time units.
• On CIFAR-100, FedRevive reaches 49.01% vs 43.13% for FedBuff; abstract claims up to 38.4% faster training and up to 16.5% higher final accuracy.
• On 20News, time-to-target is 6.1 vs 9.9 for FedBuff; FedRevive is slightly below SyncFL in final accuracy (67.09 vs 68.46).

Surprising findings

• Constant high KD mixing weight (0.75 or 1) lowers long-run accuracy, showing client updates must still be accumulated.
• Centralized training on only the LM-generated text snippets reaches 48.08% on 20News, indicating snippets are not a meaningful task-data leak.

Limitations

• Server-side DFKD requires a generator and extra compute; overhead analysis relies on simulated non-blocking server operations.
• Generator design is modality-specific; text variant uses soft prompts rather than a generator, and the paper suggests pretrained generative models as future work.
• Class-proportion proxies are coarse (KL around 0.4-0.6), and the oracle variant uses an auxiliary dataset that is unrealistic in FL.

Experimental setup

Datasets: CIFAR-10 CIFAR-100 FEMNIST 20 Newsgroups

Benchmarks: CIFAR-10 CIFAR-100 FEMNIST 20 Newsgroups

Metrics: test accuracy time-to-target accuracy (simulated wall-clock time)

Baselines: FedAvg (synchronous) FedBuff AFL down-weighting (AsyncDW) Data-Driven-FedRevive (oracle with public data)

Builds on: FedBuff FedAvg Reptile data-free knowledge distillation ResNet

Closest papers by meaning

• Efficient Asynchronous Federated Learning with Prospective Momentum Aggregation and Fine-Grained Correction (AAAI 2024 · similarity 0.91)
• Efficient Federated Learning via Clients-to-Server Knowledge Distillation (Student Abstract) (AAAI 2025 · similarity 0.90)
• FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-Aware Model Update (AAAI 2024 · similarity 0.90)
• Momentum-Driven Adaptivity: Towards Tuning-Free Asynchronous Federated Learning (ICML 2025 · similarity 0.89)
• Efficient Federated Learning against Heterogeneous and Non-stationary Client Unavailability (NeurIPS 2024 · similarity 0.88)
• FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning (NeurIPS 2025 · similarity 0.88)