Research map / Federated learning and privacy / Personalized and heterogeneous FL
Federated Learning with Extremely Noisy Clients via Negative Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedNed, which identifies extremely noisy FL clients via MC-dropout uncertainty and uses their models as 'bad teachers' in a negative distillation step on a public dataset, with pseudo-label training for those clients.
Problem
Federated learning with extremely noisy clients (noise ratios above 90%) breaks the mild-noise assumption of re-weighting-based aggregation methods, so models from such clients degrade the global model.
Approach
Instead of re-weighting or discarding extremely noisy clients, identified clients are used in a negative knowledge distillation loss that pushes the global model's predictions away from their outputs, while they also train pseudo-label models that can rejoin aggregation.
Key results
- On CIFAR-10 with Beta(0.1,0.1) noise and Dirichlet 0.7, FedNed reaches 82.83% vs. 80.25% for the second-best FedNoRo.
- Ablation on CIFAR-10 with 5 extremely noisy clients: identification alone raises accuracy from 74.26% to 79.95%; full FedNed reaches 82.07%.
- FedNed keeps accuracy stable as the number of extremely noisy clients grows, while baselines decline significantly (per authors).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add EN-client identification and aggregate only identified mildly noisy clients (ablation) | FedNed · CIFAR-10, 20 clients, five 99% noisy clients | accuracy: 74.26 -> 79.95 | Helped |
| Add negative distillation (ablation) | FedNed with EN-client identification · CIFAR-10, 20 clients, five 99% noisy clients | accuracy: 79.95 -> 81.91 | Helped |
| Add local pseudo-labeling for identified EN clients (ablation) | FedNed with EN-client identification · CIFAR-10, 20 clients, five 99% noisy clients | accuracy: 79.95 -> 81.30 | Helped |
| Use all three components: EN-client identification, negative distillation, and local pseudo-labeling (ablation) | FedNed · CIFAR-10, 20 clients, five 99% noisy clients | accuracy: 82.07 | Helped |
| Use FedNed instead of the strongest listed baseline | CIFAR-10, Beta (0.1, 0.1), Dirichlet 0.7 · CIFAR-10, Beta (0.1, 0.1), Dirichlet 0.7 | accuracy: 80.25 -> 82.83 | Helped |
| Use FedNed instead of the strongest listed baseline | CIFAR-10, Beta (0.1, 0.3), Dirichlet 10 · CIFAR-10, Beta (0.1, 0.3), Dirichlet 10 | accuracy: 85.10 -> 86.84 | Helped |
| Use FedNed instead of FedCorr | CIFAR-100, Beta (0.3, 0.5), Dirichlet 10 · CIFAR-100, Beta (0.3, 0.5), Dirichlet 10 | accuracy: 51.23 -> 49.68 | Hurt |
Surprising findings
- Discarding extremely noisy client models outperforms re-weighting them with small weights, which challenges the prevailing re-weighting paradigm in noisy-label FL.
Limitations
- Performance degenerates to FedAvg when no extremely noisy clients exist, per the authors.
- Requires an unlabeled public dataset on the server, and the uncertainty threshold (0.12) was tuned by observing the uncertainty histogram.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet MNIST
Benchmarks: CIFAR-10 CIFAR-100
Metrics: test accuracy
Baselines: FedAvg FedProx SCAFFOLD FedDyn FedCorr RoFL FedNoRo
Builds on: FedAvg FedDF FedNoRo MC dropout
Closest papers by meaning
- FedDiv: Collaborative Noise Filtering for Federated Learning with Noisy Labels (AAAI 2024 · similarity 0.91)
- FedClean: A General Robust Label Noise Correction for Federated Learning (ICML 2025 · similarity 0.91)
- FedFixer: Mitigating Heterogeneous Label Noise in Federated Learning (AAAI 2024 · similarity 0.89)
- NoT: Federated Unlearning via Weight Negation (CVPR 2025 · similarity 0.88)
- Byzantine-Robust Federated Learning: Impact of Client Subsampling and Local Updates (ICML 2024 · similarity 0.87)
- Efficient Federated Learning via Clients-to-Server Knowledge Distillation (Student Abstract) (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.