Research map / Federated learning and privacy / Communication-efficient FL

FedAdaVR: Adaptive Variance Reduction for Robust Federated Learning under Limited Client Participation

NeurIPS 2026 · Federated learning and privacy › Communication-efficient FL · approach: Client participation analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FedAdaVR, a federated learning method that combines a SAGA-style server-side variance reduction (storing each client's latest update) with adaptive server optimisers, plus FedAdaVR-Quant which stores those updates quantised to cut memory.

Problem

Partial client participation introduces a large error in federated learning, and existing variance-reduced methods like FedVARP use a fixed server learning rate, converging slowly under severe label skew.

Approach

Integrates SAGA-style server-side variance reduction, which reuses each client's most recent stored update for absent clients, with coordinate-wise adaptive server optimisers (Adagrad, Adam, Adabelief, Yogi, Lamb), and stores the stored updates in quantised form to reduce server memory.

Key results

• FedAdaVR reaches 45% accuracy on CIFAR-10 (LQ-1) in 163,689 s versus 232,336 s for FedAvg, roughly a 30% reduction in end-to-end time.
• FedAdaVR-Quant cuts server memory for stored client updates by 50%, 75%, and 87.5% (FP16, Int8, Int4) while maintaining comparable accuracy.
• The paper reports that FedAdaVR consistently outperforms baselines in nearly all settings, with 1% participation on MNIST/FMNIST, 2% on CIFAR-10, and under 1% on Shakespeare.

Surprising findings

• FedAdaVR-Quant sometimes performs better than the non-quantised version, which the authors attribute to random quantisation error.

Limitations

• Quantisation study covers only three deterministic linear schemes (FP16, Int8, Int4); no formal bound for the quantised variant.
• Hyperparameter tuning across methods is non-exhaustive, so further gains for baselines are possible.
• Evaluated only on small vision and Shakespeare benchmarks with small architectures.

Experimental setup

Datasets: MNIST FMNIST CIFAR-10 Shakespeare

Benchmarks: MNIST FMNIST CIFAR-10 Shakespeare

Metrics: accuracy training loss communication rounds wall-clock time server memory

Baselines: FedAvg FedProx SCAFFOLD FedNova FedAdam FedAdagrad FedYogi MIFA FedVARP

Builds on: FedVARP FedOpt SAGA MIFA AdaLVR

Closest papers by meaning

• Federated Learning with Sample-level Client Drift Mitigation (AAAI 2025 · similarity 0.91)
• FedImpro: Measuring and Improving Client Update in Federated Learning (ICLR 2024 · similarity 0.90)
• Efficient Federated Learning against Heterogeneous and Non-stationary Client Unavailability (NeurIPS 2024 · similarity 0.90)
• Adaptive Federated Learning with Auto-Tuned Clients (ICLR 2024 · similarity 0.90)
• Communication-Efficient Federated Learning with Accelerated Client Gradient (CVPR 2024 · similarity 0.89)
• FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning (NeurIPS 2025 · similarity 0.89)