Research map / Image generation / Text-to-image diffusion
Synthesize Privacy-Preserving High-Resolution Images via Private Textual Intermediaries
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Synthesizes high-resolution DP images by captioning private images, running a modified Aug-PE over text with image-space voting, and rendering the DP captions with off-the-shelf text-to-image models.
Problem
Generating high-fidelity, high-resolution differentially private synthetic images, where DP fine-tuning is costly and proprietary models cannot be fine-tuned, and prior Private Evolution struggles on fine-grained images.
Approach
Shift DP synthesis into the text domain: caption private images, evolve DP captions with Private Evolution using a novel Image Voting step (votes cast by nearest-neighbor matching between private image embeddings and images generated from candidate captions), then render with a text-to-image model. Requires only inference, no model training.
Key results
- On LSUN Bedroom at epsilon=1.0, the method reaches FID 26.39 (main table) versus 40.36 for Private Evolution and 31.76 for DP fine-tuning.
- On MM-CelebA-HQ at epsilon=1.0, the method reports FID 33.41 (LDM backbone) versus 57.01 for DP fine-tuning, while the abstract cites 33.27.
- The method underperforms Private Evolution on the Cat dataset (FID 106.15 vs 64.62 at epsilon=1.0), which the authors attribute to low dataset diversity.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace Text Voting with Image Voting (ablation) | SPTI · LSUN Bedroom, Cat, European Art, and Wave-ui-25k | FID: LSUN Bedroom: 39.82 -> 27.92 (ε=10), 37.83 -> 29.50 (ε=5), 41.17 -> 38.00 (ε=1); Cat: 110.51 -> 101.57 (ε=10), 106.07 -> | Helped |
| replace Text Voting with Image Voting (ablation) | SPTI · Cat (ε=1) | FID: 106.02 -> 106.15 | Hurt |
| replace Text Voting with Image Voting (ablation) | SPTI · Wave-ui-25k (ε=5, ε=1) | FID: 92.11 -> 42.02; 95.77 -> 70.87 | Helped |
| use SPTI instead of PE image | DP image synthesis · LSUN Bedroom, European Art, Wave-ui-25k, and Sprite Fright (ε=1) | FID: 40.36 -> 26.39; 76.50 -> 57.64; 50.45 -> 35.18; 197.13 -> 157.31 | Helped |
| use SPTI instead of PE image | DP image synthesis · Cat (ε=1) | FID: 64.62 -> 106.15 | Hurt |
| use SPTI instead of PE image | DP image synthesis · MM-CelebA-HQ (ε=1) | FID: 50.45 -> 33.41 | Helped |
| change caption/LLM and diffusion API backends (ablation) | SPTI · LSUN Bedroom (ε=1) | FID: 26.71; 25.42; 30.66; 26.65; 24.44; 31.28 | No clear effect |
| use Imagen3 instead of SDXL-Turbo for final image generation (ablation) | SPTI · LSUN Bedroom (ε=1) | FID: 26.71 -> 26.58 | Helped |
| use Imagen3 instead of SDXL-Turbo for final image generation (ablation) | SPTI · Wave-ui-25k (ε=1) | FID: 36.52 -> 72.13 | Hurt |
| use DALLE3 instead of SDXL-Turbo for final image generation (ablation) | SPTI · LSUN Bedroom (ε=1) | FID: 26.71 -> 29.96 | Hurt |
| use DALLE3 instead of SDXL-Turbo for final image generation (ablation) | SPTI · Wave-ui-25k (ε=1) | FID: 36.52 -> 150.68 | Hurt |
| use SPTI rather than PE to generate downstream training data | WRN-40-4 on CelebA · CelebA Wearing_Lipstick classification | test accuracy | Helped |
Surprising findings
- Image voting is not uniformly better: Private Evolution beats the method on the Cat dataset and on small or low-diversity bird subsets.
- Changing the text-to-image backend affects FID substantially on some datasets; Imagen3 gave 72.13 on Wave-ui-25k versus 36.52 with SDXL-Turbo.
Limitations
- Performance is bounded by the coverage of the captioning and text-to-image models on underrepresented domains.
- Higher compute and API cost than PE, with roughly 123M tokens for 10 iterations on 10,000 images.
- Underperforms PE on low-diversity datasets such as Cat and bird subsets.
Experimental setup
Datasets: LSUN Bedroom Cat European Art Wave-ui-25k MM-CelebA-HQ Sprite Fright CelebA ImageNet100
Benchmarks: LSUN Bedroom Cat European Art Wave-ui-25k MM-CelebA-HQ Sprite Fright CelebA
Metrics: FID test accuracy
Baselines: Private Evolution DP fine-tuning
Builds on: Private Evolution Aug-PE DP-SGD Gaussian mechanism
Closest papers by meaning
- PCEvolve: Private Contrastive Evolution for Synthetic Dataset Generation via Few-Shot Private Data and Generative APIs (ICML 2025 · similarity 0.89)
- Differentially Private Synthetic Data via Foundation Model APIs 2: Text (ICML 2024 · similarity 0.88)
- Differentially Private Synthetic Data via Foundation Model APIs 1: Images (ICLR 2024 · similarity 0.88)
- Private Evolution Converges (NeurIPS 2025 · similarity 0.87)
- SynBench: A Benchmark for Differentially Private Text Generation (NeurIPS 2026 · similarity 0.86)
- Differentially Private Representation Learning via Image Captioning (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.