Research map / Federated learning and privacy / Privacy attacks and secure learning
Recovering Private Text in Federated Learning of Language Models
Samyak Gupta, Yangsibo Huang, Zexuan Zhong, Tianyu Gao, Kai Li, Danqi Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| start from pretrained model instead of random init (ablation) | FILM attack · WikiText-103 | ROUGE-L: 0.41 (pretrained, t=0) -> 0.53 (trained) | Improved |
| apply attack repeatedly (1 to 100 iterations) (ablation) | FILM multi-sentence · WikiText-103 b=16 | recall: 0.03 -> 0.34 | Improved |
| freeze word embeddings in training from scratch | GPT-2 perplexity · WikiText-103 | perplexity: 27.31 -> 118.69 | Hurt |
| freeze word embeddings when fine-tuning pretrained GPT-2 | GPT-2 perplexity · WikiText-103 | perplexity: 11.40 -> 11.48 | No clear effect |
| freeze word embeddings | FILM defense · WikiText-103 | attack precision/recall: 1.00 -> 0.00 | Improved |
| gradient pruning up to 0.9999 | FILM defense · WikiText-103 | attack recall: 1.00 -> 0.90 | No clear effect |
| DPSGD with epsilon 1 | FILM defense · WikiText-103 | attack recall: 1.00 -> 0.00 | Improved |
| increase beam size (ablation) | FILM attack · WikiText-103 | ROUGE-L | Improved |
| add 2-gram repetition penalty (ablation) | FILM attack · WikiText-103 | ROUGE-L | Improved |
| gradient-norm weight beta=1 in reordering score vs beta=0 or 50 (ablation) | FILM attack · WikiText-103 | ROUGE-L | Improved |
| add token reordering stage (ablation) | FILM attack · WikiText-103 | ROUGE-L | Improved |
Closest papers by meaning
- DAGER: Exact Gradient Inversion for Large Language Models (NeurIPS 2024 · similarity 0.90)
- PrE-Text: Training Language Models on Private Federated Data in the Age of LLMs (ICML 2024 · similarity 0.87)
- Gradient Inversion of Multimodal Models (ICML 2025 · similarity 0.87)
- Recovering Labels from Local Updates in Federated Learning (ICML 2024 · similarity 0.86)
- FedRW: Efficient Privacy-Preserving Data Reweighting for Enhancing Federated Learning of Language Models (NeurIPS 2025 · similarity 0.86)
- Thinking Forward: Memory-Efficient Federated Finetuning of Language Models (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
