Research map / Image generation / Text-to-image diffusion
Membership Inference on Text-to-Image Diffusion Models via Conditional Likelihood Discrepancy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CLiD, a query-based membership inference method for text-to-image diffusion models that uses the discrepancy between conditional and unconditional ELBO-based likelihoods, outperforming prior attacks in fine-tuning and pretraining settings.
Problem
Membership inference on text-to-image diffusion models is ineffective under realistic training steps and common data augmentation, since existing methods mainly measure per-image overfitting and ignore the image-text conditional distribution.
Approach
Identifies a conditional overfitting phenomenon (models overfit p(x|c) more than p(x)) and derives an indicator, the conditional likelihood discrepancy, estimated via Monte Carlo ELBO differences between text-conditioned and reduced/null-text conditioned denoising losses; combined either via a threshold or an XGBoost feature-vector classifier.
Key results
- Under the real-world fine-tuning setting on MS-COCO, CLiD_vec reaches 89.52% ASR and 96.30 AUC versus best baseline SecMI around 60.94% ASR and 65.40 AUC.
- Under over-training on MS-COCO, CLiD_th achieves 99.08% ASR, 99.94 AUC and 99.12% TPR@1%FPR.
- With default data augmentation removed on MS-COCO, CLiD_th achieves 96.76 ASR and 99.47 AUC, while baselines like PFAMI drop substantially under augmentation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use CLiD_th instead of the strongest baseline, SecMI | Stable Diffusion v1-4 membership inference · MS-COCO, real-world training | ASR: 60.94 -> 88.88 | Helped |
| use CLiD_vec instead of the strongest baseline, SecMI | Stable Diffusion v1-4 membership inference · Flickr, real-world training | AUC: 63.85 -> 95.33 | Helped |
| add conditional likelihood discrepancy to the likelihood term (ablation) | CLiD_th on MS-COCO · Real-world training | AUC | Helped |
| use both the discrepancy and likelihood terms instead of either term alone (ablation) | CLiD_th on MS-COCO · Real-world training | AUC | Helped |
| increase Monte Carlo sampling when M=N (ablation) | CLiD_th on MS-COCO · Real-world training | AUC | Helped |
| increase Monte Carlo sampling beyond 3 per term (ablation) | CLiD_th on MS-COCO · Real-world training | AUC | No clear effect |
| replace null-only likelihood discrepancy with condition reduction (ablation) | CLiD_th on MS-COCO · Real-world training | AUC: 93.60 -> 96.13 | Helped |
| use importance clipping rather than simple clipping for condition reduction (ablation) | CLiD_th on MS-COCO · Real-world training | AUC: 95.90 -> 96.13 | Helped |
| use importance clipping rather than Gaussian noise for condition reduction (ablation) | CLiD_th on MS-COCO · Real-world training | AUC: 94.79 -> 96.13 | Helped |
| add default Random-Crop and Random-Flip augmentation during training | CLiD_th on MS-COCO · Real-world training | AUC: 99.47 -> 96.13 (-3.34) | Hurt |
| add default data augmentation during training | PFAMI on MS-COCO · Real-world training | AUC: 87.05 -> 60.39 (-26.66) | Hurt |
| rephrase training captions as an adaptive defense | CLiD_th on MS-COCO · Real-world training | AUC: 96.13 -> 93.83 | Hurt |
Surprising findings
- Existing membership inference methods for diffusion models yield near-random results under realistic training steps and default augmentation, with the best baseline ASR around 65%.
- Shuffling image-text pairs as an adaptive defense degrades utility heavily (FID rises by 5.09), while text rephrasing or word deletion barely reduces CLiD effectiveness.
Limitations
- Pretraining evaluation is limited because few open-source text-to-image models are available, so results there are less extensive.
- Advantage over baselines in the pretraining setting is less evident than in fine-tuning.
- Requires a shadow model and auxiliary data from the same distribution to set threshold and classifier; query count of 15 per sample.
Experimental setup
Datasets: Pokemon MS-COCO Flickr LAION-Aesthetics v2 5+ LAION-2B MultiTranslated
Benchmarks: Pokemon MS-COCO Flickr LAION-Aesthetics v2 5+ / LAION-2B MultiTranslated
Metrics: ASR AUC TPR@1%FPR FID
Baselines: Loss-based inference SecMI PIA PFAMI Monte Carlo estimation
Builds on: LiRA SecMI PFAMI DDPM
Closest papers by meaning
- DIAGNOSIS: Detecting Unauthorized Data Usages in Text-to-image Diffusion Models (ICLR 2024 · similarity 0.90)
- The Illusion of Unlearning: The Unstable Nature of Machine Unlearning in Text-to-Image Diffusion Models (CVPR 2025 · similarity 0.89)
- Towards Black-Box Membership Inference Attack for Diffusion Models (ICML 2025 · similarity 0.88)
- Detecting, Explaining, and Mitigating Memorization in Diffusion Models (ICLR 2024 · similarity 0.88)
- How Diffusion Models Memorize (NeurIPS 2026 · similarity 0.88)
- Mining your own secrets: Diffusion Classifier Scores for Continual Personalization of Text-to-Image Diffusion Models (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.