Research map / Federated learning and privacy / Differential privacy

SynBench: A Benchmark for Differentially Private Text Generation

NeurIPS 2026 · Federated learning and privacy › Differential privacy · approach: Differentially private synthetic data · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces SynBench, a benchmark and membership-inference-based audit framework for differentially private synthetic text generation across nine domain-specific datasets, finding that LLM pre-training contamination inflates utility/fidelity and can make empirical privacy leakage exceed claimed DP bounds.

Problem

Existing DP synthetic text generation methods are compared with inconsistent evaluation setups and datasets that may be contaminated by LLM pre-training data, and their DP guarantees are rarely verified empirically with audits.

Approach

A unified benchmark of nine domain-specific datasets with utility, fidelity (MAUVE, NER and length divergence) and a black-box membership inference audit built on outlier targets and n-gram reference models, combined with a pre-training leakage analysis via Infini-gram against RedPajama.

Key results

• Without privacy, DP-Gen rescaled improvements range from -5.21% to 61.78% over random/majority baselines, versus 17.83-48.61% max improvements under epsilon<=4 for DP methods.
• On PsyTAR, excluding 25%/50%/75% of flagged contaminated samples monotonically lowers DP-Gen F1 at epsilon=2 (43.75 -> 43.28 -> 42.52).
• AUG-PE on PsyTAR, DMSAFN and n2c2'08 at epsilon=0.5 shows empirical epsilon exceeding the theoretical bound at certain FPR ranges despite low overall AUC.

Surprising findings

• Synthetic data quality is overestimated when generator LLMs were pre-trained on portions of the nominally private data, so DP guarantees can be invalidated in practice.
• Tighter epsilon does not always reduce empirical MIA leakage; on PsyTAR AUC rises from 0.39 at epsilon=4 to 0.58 at epsilon=0.5.

Limitations

• Full audit protocol is resource-intensive (100 splits plus synthetic generation).
• Pre-training leakage estimated with RedPajama as a proxy because the LLaMA pre-training corpora are not public.
• Outlier-based attack targets represent a worst-case scenario and may not reflect typical real-world targets.

Experimental setup

Datasets: HoC N2C2'08 PsyTAR MIMIC DMSAFN Lc37Fnd ZsTfns AsyLex EurLex RedPajama

Benchmarks: HoC N2C2'08 PsyTAR MIMIC DMSAFN Lc37Fnd ZsTfns AsyLex EurLex

Metrics: F1 MAUVE NER distribution divergence text length divergence AUC empirical epsilon perplexity Self-BLEU Distinct-n

Baselines: AUG-PE DP-transformers (DP-Gen) DP-Diffusion (DP-Dfs)

Builds on: AUG-PE DP-SGD MAUVE RMIA LiRA

Closest papers by meaning

• Struct-Bench: A Benchmark for Differentially Private Structured Text Generation (NeurIPS 2025 · similarity 0.93)
• InvisibleInk: High-Utility and Low-Cost Text Generation with Differential Privacy (NeurIPS 2025 · similarity 0.91)
• Differentially Private Synthetic Data via Foundation Model APIs 2: Text (ICML 2024 · similarity 0.91)
• Synthesizing Privacy-Preserving Text Data via Finetuning *without* Finetuning Billion-Scale LLMs (ICML 2025 · similarity 0.91)
• PrE-Text: Training Language Models on Private Federated Data in the Age of LLMs (ICML 2024 · similarity 0.90)
• The Canary’s Echo: Auditing Privacy Risks of LLM-Generated Synthetic Text (ICML 2025 · similarity 0.90)