Research map / Federated learning and privacy / Differential privacy

InvisibleInk: High-Utility and Low-Cost Text Generation with Differential Privacy

NeurIPS 2025 · Federated learning and privacy › Differential privacy · approach: Differentially private synthetic data

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes InvisibleInk, a DP long-form text generation method that treats LLM next-token sampling as an exponential mechanism over clipped logits, using a relative clipping function (DClip) and a Top-k+ truncated decoding set, achieving ~8x lower compute than prior white-box DP generation at equal privacy and utility.

Problem

Generating long-form text from LLMs conditioned on sensitive reference texts (e.g., RAG or synthetic data) with rigorous example-level differential privacy, while keeping utility high and computation cost low.

Approach

Clip only the difference between private and public logits (DClip) to reduce sensitivity, and sample tokens without privacy cost from a tight superset of the public top-k set (Top-k+), so that the temperature depends only on C/B and the vocabulary restriction is data-independent.

Key results

• At (ε=10, δ=1e-6) on MIMIC, InvisibleInk reaches MAUVE ~71-74% with 8-16 next-token calls per token, versus ~70-72% for the Flemings et al. baseline at 64-128 calls (about 8x compute reduction).
• Flemings et al. produces no text at B≤31 for ε≤5 and at all B≤127 for ε=1, while InvisibleInk degrades gracefully.
• Top-k+ sampling improves MAUVE over full-vocabulary sampling at ε=10 (72% vs 65% for TinyLlama; 76% vs 62% for Llama-3.2-1B).

Measured effects reported in the paper

Surprising findings

• Baseline sensitivity was intentionally underestimated (C/B instead of 2C/B) to give baselines an advantage, and InvisibleInk still outperformed them.
• AdaPMixED produced very short outputs and its downstream classification scores were near random-level behavior, suggesting downstream metrics can be hacked by poor text.

Limitations

• Requires a trusted central aggregator with access to raw sensitive references and white-box model access.
• Guarantees cover inference only, not training/fine-tuning, and only example-level DP rather than user-level.
• Still roughly 8x overhead over non-private generation, which may be restrictive for large frontier models under tight budgets.

Experimental setup

Datasets: MIMIC-IV-Note Text Anonymization Benchmark (TAB) Yelp Reviews

Benchmarks: MIMIC-IV-Note TAB Yelp

Metrics: MAUVE perplexity gap (ΔPPL) average generation length named-entity count downstream classification accuracy

Baselines: Flemings et al. white-box DP decoding AdaPMixED AugPE

Builds on: exponential mechanism zCDP Private next-token prediction (Flemings et al.) AdaPMixED AugPE top-k sampling

Closest papers by meaning

• SynBench: A Benchmark for Differentially Private Text Generation (NeurIPS 2026 · similarity 0.91)
• Synthesizing Privacy-Preserving Text Data via Finetuning *without* Finetuning Billion-Scale LLMs (ICML 2025 · similarity 0.88)
• Cape: Context-Aware Prompt Perturbation Mechanism with Differential Privacy (ICML 2025 · similarity 0.87)
• Open LLMs are Necessary for Current Private Adaptations and Outperform their Closed Alternatives (NeurIPS 2024 · similarity 0.87)
• MERGE: Fast Private Text Generation (AAAI 2024 · similarity 0.86)
• Differentially Private Synthetic Data via Foundation Model APIs 2: Text (ICML 2024 · similarity 0.86)