Research map / Federated learning and privacy / Differential privacy
Noise-Aware Algorithm for Heterogeneous Differentially Private Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Robust-HDP, a noise-aware aggregation for heterogeneous differentially private federated learning on an untrusted server, using RPCA to estimate each client's model-update noise level and weighting clients by inverse noise variance.
Problem
Heterogeneous privacy parameters and batch/dataset sizes across clients create varying DP noise levels in model updates; existing aggregation (weights proportional to epsilon) is suboptimal and vulnerable to clients falsifying their epsilon, especially when the server is untrusted.
Approach
The untrusted server stacks client model updates into a matrix, decomposes it via Robust PCA (principal component pursuit) into low-rank and sparse parts, and uses the squared column norms of the sparse noise part as noise-variance estimates to set inverse-variance aggregation weights, without needing clients' privacy parameters.
Key results
- On MNIST with heterogeneous privacy and batch sizes, Robust-HDP achieves higher average test accuracy than WeiAvg, PFA, DPFedAvg and minimum-epsilon baselines across privacy distributions.
- Robust-HDP's estimated per-parameter aggregated noise variance closely matches the oracle optimum (e.g., 0.27 vs 0.27 and 7.62 vs 7.60 on FMNIST Dist1/Dist6).
- Under a falsified epsilon attack, WeiAvg and PFA are highly vulnerable while Robust-HDP is unaffected, since it does not use reported epsilon values.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use noise-aware RPCA aggregation instead of privacy-parameter-based aggregation | heterogeneous DPFL · MNIST, heterogeneous data split, 60 clients, Dist1–Dist9 | test accuracy | Mixed |
| Use noise-aware RPCA aggregation instead of WeiAvg | heterogeneous DPFL · MNIST, heterogeneous data split, Dist1 | test accuracy: 81.14 -> 84.84 | Helped |
| Use noise-aware RPCA aggregation instead of WeiAvg | heterogeneous DPFL · MNIST, heterogeneous data split, Dist3 | test accuracy: 84.44 -> 80.78 | Hurt |
| Use noise-aware RPCA aggregation instead of PFA | heterogeneous DPFL · MNIST, heterogeneous data split, Dist2 | test accuracy: 77.40 -> 82.78 | Helped |
| Use noise-aware RPCA aggregation instead of PFA | heterogeneous DPFL · MNIST, heterogeneous data split, Dist5 | test accuracy: 81.89 -> 81.66 | Hurt |
| Estimate client noise with RPCA and weight updates inversely by estimated noise variance | Robust-HDP · FMNIST Dist1 | normalized aggregated-update noise variance: 1.02 -> 0.27 | Helped |
| Estimate client noise with RPCA and weight updates inversely by estimated noise variance | Robust-HDP · FMNIST Dist6 | normalized aggregated-update noise variance: 28.29 -> 7.62 | Helped |
Surprising findings
- Under moderately heterogeneous data splits (max 8 labels per client, 60 clients), Robust-HDP's advantage over baselines shrinks noticeably, and WeiAvg beats it on some distributions.
Limitations
- Relies on the noise matrix being approximately sparse and on the low-rank structure of updates, which holds for i.i.d. or mildly heterogeneous data; advantage diminishes under strong heterogeneity.
- Precision of the RPCA noise estimate depends on the number of parameters p and sub-matrix size; RPCA computational cost grows with p.
Experimental setup
Datasets: MNIST FMNIST CIFAR10 CIFAR100
Benchmarks: MNIST FMNIST CIFAR10 CIFAR100
Metrics: test accuracy noise variance convergence speed
Baselines: WeiAvg PFA DPFedAvg minimum epsilon
Builds on: DPSGD FedAvg Principal Component Pursuit WeiAvg PFA
Closest papers by meaning
- Towards the Robustness of Differentially Private Federated Learning (AAAI 2024 · similarity 0.92)
- Towards Trustworthy Federated Learning with Untrusted Participants (ICML 2025 · similarity 0.90)
- Private Heterogeneous Federated Learning Without a Trusted Server Revisited: Error-Optimal and Communication-Efficient Algorithms for Convex Losses (ICML 2024 · similarity 0.90)
- Self-Driven Entropy Aggregation for Byzantine-Robust Heterogeneous Federated Learning (ICML 2024 · similarity 0.89)
- Mitigating the Privacy–Utility Trade-off in Decentralized Federated Learning via f-Differential Privacy (NeurIPS 2025 · similarity 0.89)
- Privacy-Preserving Federated Convex Optimization: Balancing Partial-Participation and Efficiency via Noise Cancellation (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.