Research map / Federated learning and privacy / Differential privacy

Private Federated Learning using Preference-Optimized Synthetic Data

ICML 2025 · Federated learning and privacy › Differential privacy · approach: Differentially private federated learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes POPri, which treats client similarity feedback for DP synthetic text as a reward and fine-tunes an LLM with DPO to generate better DP synthetic data for on-device federated learning, and releases the LargeFedBench benchmark.

Problem

Differentially private federated learning on on-device text is costly, and prior DP synthetic data methods (Private Evolution variants) rely on prompting and discard low-scoring synthetic samples, limiting data quality.

Approach

Reformulates private-client-feedback-driven synthetic data generation as LLM policy optimization: clients return DP-noised cosine-similarity scores for K×J synthetic samples, the server builds chosen/rejected preference pairs, and fine-tunes the generator LLM with DPO (LoRA) over several rounds.

Key results

• On bioRxiv at epsilon=1, POPri reaches 34.8% next-token accuracy vs 31.1% for PE and 28.2% for DP-FTRL, closing 58% of the gap between fully private and non-private baselines.
• On OpenReview at epsilon=1, POPri achieves 40.2% accuracy vs 37.0% for Aug-PE-style PE with Llama-2-7b-chat-hf.
• On Congress at epsilon=1, POPri achieves 30.4% vs 27.0% for PE.

Measured effects reported in the paper

Surprising findings

• Cosine similarity scoring works much better than the nearest-neighbor histogram used by Private Evolution, since the histogram is too sparse and yields nearly random preference pairs.
• Using the 5th-ranked sample as the rejected sample works best; the highest-ranked (rank 2) rejected sample degrades results due to DP noise swapping rankings.

Limitations

• Synthetic data quality depends on the public LLM and may inherit its biases, underrepresenting clients whose language the LLM does not cover.
• Server compute cost is high (about 13,548 GPU seconds per round) and POPri is more communication and compute-intensive than PE on clients.
• Downstream accuracy is not strongly sensitive to epsilon, and synthetic-data distributions degrade (overfit) after several rounds, requiring early stopping on validation.

Experimental setup

Datasets: LargeFedBench (bioRxiv, Congressional Speeches) PubMed OpenReview c4

Benchmarks: LargeFedBench bioRxiv LargeFedBench Congress PubMed OpenReview

Metrics: next-token prediction accuracy text classification accuracy FID

Baselines: DP-FedAvg DP-FTRL PrE-Text PE+SFT Aug-PE DP-Prompt

Builds on: Private Evolution DPO PrE-Text DP-FedAvg Aug-PE

Closest papers by meaning

• PrE-Text: Training Language Models on Private Federated Data in the Age of LLMs (ICML 2024 · similarity 0.86)
• Noise-Aware Algorithm for Heterogeneous Differentially Private Federated Learning (ICML 2024 · similarity 0.85)
• $\texttt{pfl-research}$: simulation framework for accelerating research in Private Federated Learning (NeurIPS 2024 · similarity 0.85)
• A Bayesian Approach for Personalized Federated Learning in Heterogeneous Settings (NeurIPS 2024 · similarity 0.85)
• Fine-Tuning Personalization in Federated Learning to Mitigate Adversarial Clients (NeurIPS 2024 · similarity 0.84)
• Differentially Private Synthetic Data via Foundation Model APIs 1: Images (ICLR 2024 · similarity 0.84)