Research map / Image generation / Fast diffusion sampling
DRAG: Data Reconstruction Attack using Guided Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DRAG, a guided-diffusion data reconstruction attack that uses a pre-trained Stable Diffusion prior to invert deep-layer intermediate representations of vision foundation models (CLIP, DINOv2) in split inference, plus DRAG++ with an inverse-network initialization.
Problem
Privacy risks of split inference with large vision foundation models (ViTs, CLIP, DINOv2) are underexplored, since existing data reconstruction attacks mainly target small CNNs and degrade at deep layers.
Approach
Use a pre-trained latent diffusion model as an image prior and steer its sampling with guidance from the feature-matching loss on the target IR (DSG-style spherical-constrained guidance with Adam-smoothed gradients and self-recurrence), optionally initialized by an inverse network (DRAG++).
Key results
- At CLIP-ViT-B/16 layer 12 without defense, DRAG achieves MS-SSIM 0.6735, LPIPS 0.1857, DINO 0.9331, versus best baseline rMLE MS-SSIM 0.3884 and LPIPS 0.5790 (GLASS).
- Under DISCO Config-III with adaptive attack, DRAG reaches MS-SSIM 0.4557 and DINO 0.7557 versus GLASS 0.2402 and 0.4862.
- DRAG++ improves LPIPS and DINO over the inverse network alone at deep layers, e.g., layer 12 DINO 0.9492 vs 0.6660 for IN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use guided diffusion (DRAG) instead of the strongest listed baseline | CLIP-ViT-B/16 at Layer 12 · CLIP-ViT-B/16 Layer 12 | MS-SSIM: 0.2560 -> 0.6735 | Helped |
| Use guided diffusion (DRAG) instead of the strongest listed baseline | CLIP-ViT-B/16 at Layer 12 · CLIP-ViT-B/16 Layer 12 | LPIPS: 0.6024 -> 0.1857 | Helped |
| Use guided diffusion (DRAG) instead of the strongest listed baseline | DINOv2-Base at Layer 12 · DINOv2-Base Layer 12 | MS-SSIM: 0.4375 -> 0.7581 | Helped |
| Use guided diffusion (DRAG) instead of the strongest listed baseline | DINOv2-Base at Layer 12 · DINOv2-Base Layer 12 | LPIPS: 0.4076 -> 0.1443 | Helped |
| Use guided diffusion (DRAG) instead of the strongest listed baseline | CLIP-RN50 at Block 5 · CLIP-RN50 Block 5 | MS-SSIM: 0.4432 -> 0.5206 | Helped |
| Add an inverse-network coarse reconstruction to initialize DRAG (ablation) | DRAG on CLIP-ViT-B/16 at Layer 12 · CLIP-ViT-B/16 Layer 12 | LPIPS: 0.1857 -> 0.1685 | Helped |
| Add an inverse-network coarse reconstruction to initialize DRAG (ablation) | DRAG on CLIP-ViT-B/16 at Layer 12 · CLIP-ViT-B/16 Layer 12 | MS-SSIM: 0.6735 -> 0.7257 | Helped |
| Replace Stable Diffusion v1.5 with an LSUN-bedroom diffusion prior (ablation) | DRAG on CLIP-ViT-B/16 · CLIP-ViT-B/16 Layer 12, LSUN-bedroom prior | MS-SSIM: 0.3958 | Mixed |
| Filter low-mean-absolute-value channels during DISCO distance calculation | DRAG on CLIP-ViT-B/16, DISCO Config-III · CLIP-ViT-B/16 Layer 12, DISCO Config-III | DINO: 0.3201 -> 0.7557 | Helped |
| Increase the number of denoising steps T or self-recurrence iterations k (ablation) | DRAG · Reconstruction experiments | reconstruction performance | Helped |
| Use Adam to smooth guidance gradients (ablation) | DRAG on deep-layer IR · CLIP-ViT-B/16 Layer 12 | reconstruction performance | Helped |
| Use linear or spherical linear interpolation for guidance (ablation) | DRAG · Reconstruction experiments | reconstruction performance | No clear effect |
Surprising findings
- Standard optimization-based attacks such as rMLE and LM degrade sharply at deep layers, while the diffusion-based attack remains effective.
- Token shuffling and dropping hinders reconstruction, but a learned token-position classifier partially recovers ordering, and DRAG still reconstructs recognizable features.
Limitations
- Relies on white-box access to the client model, so the threat model assumes known frozen encoder weights.
- Under DISCO defense without adaptive filtering, reconstruction quality drops sharply (MS-SSIM 0.0788 at Config-III), and performance needs an adaptive attack.
- Token-order reconstruction with shuffling is only partially effective (21.40% top-1 position accuracy).
Experimental setup
Datasets: MSCOCO FFHQ ImageNet-1K UCMerced LandUse LSUN-bedroom
Benchmarks: MSCOCO FFHQ ImageNet-1K UCMerced LandUse
Metrics: MS-SSIM LPIPS DINO ViT-S/16 similarity attack success rate ImageNet-1K classification accuracy
Baselines: rMLE LM GLASS DISCO NoPeek
Builds on: classifier guidance Universal Guidance Diffusion with Spherical Gaussian constraint GLASS rMLE DISCO
Closest papers by meaning
- Hiding Images in Diffusion Models by Editing Learned Score Functions (CVPR 2025 · similarity 0.87)
- PID: Prompt-Independent Data Protection Against Latent Diffusion Models (ICML 2024 · similarity 0.86)
- Membership Inference Attacks on Diffusion Models via Quantile Regression (ICML 2024 · similarity 0.86)
- RAPID: Retrieval Augmented Training of Differentially Private Diffusion Models (ICLR 2025 · similarity 0.86)
- Understanding Reconstruction Attacks with the Neural Tangent Kernel and Dataset Distillation (ICLR 2024 · similarity 0.86)
- TERD: A Unified Framework for Safeguarding Diffusion Models Against Backdoors (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.