Research map / Federated learning and privacy / Personalized and heterogeneous FL
FedSDR: Federated Self-Distillation with Rectification
Ziheng Ren, Zhanming Shen, Hao Wang, Ning Liu, You Song
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add FedSD self-distillation to FedAvg | FedAvg · Dolly-15K Overall Score | Overall Score: 71.21 -> 74.54 | Improved |
| add FedSD to FedAvgM | FedAvgM · Dolly-15K Overall Score | Overall Score: 68.18 -> 73.34 | Improved |
| add FedSD to FedYogi | FedYogi · Dolly-15K MMLU | MMLU: 29.92 -> 39.32 | Improved |
| add FedSD to FedProx | FedProx · Dolly-15K DROP | DROP: 37.50 -> 36.68 | Hurt |
| add FedSD to FedAvg under high heterogeneity | FedAvg · Dolly-15K alpha=0.1 | Overall: 53.18 -> 63.26 | Improved |
| self-distill client data (rewrite with model) | client data · Dolly-15K inter-client divergence | JS Divergence: 0.4074 -> 0.3611 | Improved |
| self-distill client data | client data · Dolly-15K inter-client similarity | TF-IDF cosine similarity: 0.6362 -> 0.7064 | Improved |
| add dual-stream rectification (LoRA-R, selective aggregation) to FedSD (ablation) | FedSD · Dolly-15K extended ablation | Overall: 69.05 -> 77.38 | Improved |
| upload only LoRA-R instead of both adapters (selective aggregation) (ablation) | Dual-LoRA(FedDPA) · Dolly-15K extended ablation | Overall: 68.77 -> 77.38 | Improved |
| add FedSDR rectification versus FedSD on faithfulness (ablation) | FedSD · Dolly-15K extended ablation | Faithfulness: 66.67 -> 74.33 | Improved |
| scale clients from 8 to 40 with FedSDR | FedSDR · Dolly-15K number of clients | Overall: 75.24 -> 77.70 | Improved |
| use FedSDR instead of FedAvg on Qwen2.5-7B-Instruct mixed data | FedAvg · GSM8K (mixed 5-domain FL) | accuracy: 66.79 -> 78.32 | Improved |
Closest papers by meaning
- Beyond Parameter Aggregation: Semantic Consensus for Federated Fine-Tuning of LLMs (NeurIPS 2026 · similarity 0.90)
- Ferret: Federated Full-Parameter Tuning at Scale for Large Language Models (ICML 2025 · similarity 0.90)
- Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes (ICML 2024 · similarity 0.90)
- FlowerTune: A Cross-Domain Benchmark for Federated Fine-Tuning of Large Language Models (NeurIPS 2025 · similarity 0.89)
- FedBPT: Efficient Federated Black-box Prompt Tuning for Large Language Models (ICML 2024 · similarity 0.89)
- Thinking Forward: Memory-Efficient Federated Finetuning of Language Models (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
