Research map / Image generation / Autoregressive and latent generation
Towards Black-Box Membership Inference Attack for Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ReDiffuse, a black-box membership inference attack on diffusion models that uses only the image-variation API by averaging repeated variations and comparing with the original image.
Problem
Membership inference against proprietary diffusion models, where prior MIA methods require access to internal U-Net/transformer noise predictions and commercial models expose only black-box APIs.
Approach
Members yield unbiased noise predictions, so averaging multiple variation-API outputs of a training image lands close to the original; a distance (SSIM or a learned ResNet-18 on difference vectors) thresholded against non-members classifies membership without U-Net access.
Key results
- On DDIM CIFAR-10, ReDiffuse reaches AUC 0.96 and ASR 0.91 versus 0.95/0.90 for the best prior baseline (SecMI/PIA), while not requiring U-Net access.
- On Diffusion Transformer ImageNet 128x128, ReDiffuse reaches AUC 0.98 and TP 44.1 at 1% FPR, outperforming all baselines integrated into DiT.
- On Stable Diffusion v1-4 (Laion5), ReDiffuse reaches AUC 0.81 versus 0.70 for the best baseline; on DALL-E 2's variation API, AUC is 88.3 with L2 distance on a 30-painting member set.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ReDiffuse instead of the strongest baseline (PIA) | DDIM membership inference attack · CIFAR10 | AUC: 0.95 -> 0.96 | Helped |
| Use ReDiffuse instead of the strongest baseline (PIA) | DDIM membership inference attack · CIFAR100 | AUC: 0.96 -> 0.98 | Helped |
| Use ReDiffuse instead of the strongest baseline (PIA) | DDIM membership inference attack · STL10 | AUC: 0.94 -> 0.96 | Helped |
| Use ReDiffuse instead of PIA | Diffusion Transformer membership inference attack · ImageNet 128 × 128 | AUC: 0.97 -> 0.98 | Helped |
| Use ReDiffuse instead of SecMI | Diffusion Transformer membership inference attack · ImageNet 256 × 256 | AUC: 0.88 -> 0.97 | Helped |
| Use ReDiffuse instead of PIA | Stable Diffusion membership inference attack · Laion5 | AUC: 0.70 -> 0.81 | Helped |
| Use ReDiffuse instead of PIA | Stable Diffusion membership inference attack · Laion5 with BLIP | AUC: 0.73 -> 0.82 | Helped |
| Average multiple independent variation API outputs instead of using a single output (ablation) | ReDiffuse on DDIM CIFAR10 · CIFAR10 | accuracy | Helped |
| Average multiple independent variation API outputs instead of using a single output (ablation) | ReDiffuse on Diffusion Transformer ImageNet · ImageNet | accuracy | Helped |
| Average multiple independent variation API outputs instead of using a single output (ablation) | ReDiffuse on Stable Diffusion · LAION-5B | AUC | No clear effect |
| Vary the diffusion step across a moderate range (ablation) | ReDiffuse on DDIM · CIFAR10 | attack performance | No clear effect |
| Use an excessively high diffusion step (ablation) | ReDiffuse · Diffusion models | accuracy | Hurt |
Surprising findings
- For Stable Diffusion, averaging multiple variations does not improve performance because 512x512 reconstructions are already stable.
- The PIAN baseline performs near chance on Stable Diffusion (AUC 0.56) and on DiT ImageNet 256 TP is lower than PIA's, indicating baseline instability across setups.
Limitations
- Accuracy declines when the diffusion step t is excessively high, requiring a moderate t.
- DALL-E 2 experiment uses an assumed member set of famous paintings and a non-member set generated by Stable Diffusion 3, so ground truth is uncertain and distribution shift is possible.
- Distance function for DDIM/DiT relies on a trained ResNet-18 classifier, requiring labeled member/non-member data.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 STL10-Unlabeled ImageNet LAION-5B COCO2017-val famous paintings (DALL-E 2 experiment)
Benchmarks: CIFAR-10 CIFAR-100 STL10-Unlabeled ImageNet 128x128 ImageNet 256x256 LAION5 / COCO2017-val (Stable Diffusion) DALL-E 2 variation API famous paintings
Metrics: AUC ASR TP at 1% FPR
Baselines: Loss SecMI PIA PIAN
Builds on: SecMI PIA PIAN DDIM Stable Diffusion
Closest papers by meaning
- Membership Inference Attacks on Diffusion Models via Quantile Regression (ICML 2024 · similarity 0.92)
- Toward effective protection against diffusion-based mimicry through score distillation (ICLR 2024 · similarity 0.89)
- An Efficient Membership Inference Attack for the Diffusion Model by Proximal Initialization (ICLR 2024 · similarity 0.89)
- Membership Inference on Text-to-Image Diffusion Models via Conditional Likelihood Discrepancy (NeurIPS 2024 · similarity 0.88)
- Intriguing Properties of Diffusion Models: An Empirical Study of the Natural Attack Capability in Text-to-Image Generative Models (CVPR 2024 · similarity 0.88)
- CDI: Copyrighted Data Identification in Diffusion Models (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.