AI Paper Atlas

Research map / Federated learning and privacy / Privacy attacks and secure learning

Recovering Private Text in Federated Learning of Language Models

Samyak Gupta, Yangsibo Huang, Zexuan Zhong, Tianyu Gao, Kai Li, Danqi Chen

NeurIPS 2022 · Federated learning and privacy › Privacy attacks and secure learning · approach: Gradient inversion attacks · 21 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
start from pretrained model instead of random init (ablation)FILM attack · WikiText-103ROUGE-L: 0.41 (pretrained, t=0) -> 0.53 (trained)Improved
apply attack repeatedly (1 to 100 iterations) (ablation)FILM multi-sentence · WikiText-103 b=16recall: 0.03 -> 0.34Improved
freeze word embeddings in training from scratchGPT-2 perplexity · WikiText-103perplexity: 27.31 -> 118.69Hurt
freeze word embeddings when fine-tuning pretrained GPT-2GPT-2 perplexity · WikiText-103perplexity: 11.40 -> 11.48No clear effect
freeze word embeddingsFILM defense · WikiText-103attack precision/recall: 1.00 -> 0.00Improved
gradient pruning up to 0.9999FILM defense · WikiText-103attack recall: 1.00 -> 0.90No clear effect
DPSGD with epsilon 1FILM defense · WikiText-103attack recall: 1.00 -> 0.00Improved
increase beam size (ablation)FILM attack · WikiText-103ROUGE-LImproved
add 2-gram repetition penalty (ablation)FILM attack · WikiText-103ROUGE-LImproved
gradient-norm weight beta=1 in reordering score vs beta=0 or 50 (ablation)FILM attack · WikiText-103ROUGE-LImproved
add token reordering stage (ablation)FILM attack · WikiText-103ROUGE-LImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy