Research map / Image generation / Autoregressive and latent generation
Aligned Datasets Improve Detection of Latent Diffusion-Generated Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds a latent-diffusion fake-image detector by training a ResNet-50 to separate real images from their VAE reconstructions (no denoising), yielding aligned real/fake data that reduces spurious resolution cues and generalizes across LDMs.
Problem
Fake image detectors for latent diffusion model (LDM) outputs are trained on real/fake datasets that are misaligned in resolution, aspect ratio and content, so detectors learn spurious correlations (e.g., resizing artifacts) and fail under post-processing.
Approach
Create fake images by passing real images only through the LDM autoencoder (encoder then decoder, no UNet/denoising), so each fake is nearly identical to its real counterpart in size, aspect ratio and content, forcing the detector to learn VAE decoder artifacts; optionally batch-synchronize real and reconstructed pairs (Ours-Sync).
Key results
- Ours-Sync reaches 99.48% accuracy on Playground vs 82.13% for Corvi on clean images, and 76.36% vs 24.27% on post-processed Playground images.
- With only 1k real/1k fake training pairs, Ours achieves 83.37% TPR@5FPR vs 46.51% for Corvi.
- Detector trained only on OpenGL shader images (no natural images) outperforms Corvi on post-processed Kandinsky, Playground, PixelArt-alpha and LCM images.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train on real images paired at the dataset level with their LDM VAE reconstructions instead of iteratively generated fakes | ResNet-50 fake-image detector · Playground | accuracy: 82.13 -> 94.85 | Helped |
| Train on real images paired at the dataset level with their LDM VAE reconstructions instead of iteratively generated fakes | ResNet-50 fake-image detector · Playground (post-processed) | accuracy: 24.27 -> 61.25 | Helped |
| Add batch-level pairing and identical augmentations for each real image and its reconstruction (ablation) | Ours detector · Playground (post-processed) | accuracy: 61.25 -> 76.36 | Helped |
| Train on fewer aligned examples rather than Corvi's training data (ablation) | Ours detector · 1k / 1k training data | TPR@5FPR: 46.51 -> 83.37 | Helped |
| Train on shader-generated images and their VAE reconstructions rather than natural-image training data (ablation) | Ours (shaders) · Playground (post-processed) | accuracy: +28.54 | Helped |
| Use shader-generated training images instead of natural-image training images (ablation) | Ours (shaders) · Post-processed fake images | accuracy | Hurt |
| Replace ImageNet initialization with DINO initialization (ablation) | ResNet-50 detector trained on aligned data · SD | average precision: 99.40 -> 99.52 | Helped |
| Replace ImageNet ResNet-50 with ImageNet ViT-B/16 (ablation) | Detector trained on aligned data · Playground | average precision: 98.54 -> 82.01 | Hurt |
| Downsample input images to 256 x 256 (ablation) | Ojha-LDM · SD images | accuracy: 69.56 -> 29.83 | Hurt |
| Downsample input images to 256 x 256 (ablation) | Ojha-LDM · Real images | accuracy: 54.16 -> 83.58 | Helped |
| Use the aligned reconstruction training approach on images from a model with a major VAE architecture difference | Ours detector · FLUX.1-dev | accuracy: 9.59% | Hurt |
Surprising findings
- A detector trained purely on algorithmically generated shader images, with no natural real images or denoised fakes, beats Corvi on post-processed fake images.
- Ojha-LDM was found to associate downsampling with real images, and its real accuracy rises from 54.16% to 83.58% when images are downsampled.
Limitations
- Fails to be robust to .webp compression because reconstructions do not inherit the real images' webp artifacts.
- Struggles with major VAE architecture differences, e.g., FLUX.1-dev with a 16-channel latent (accuracy 3.18%, 9.59%, 25.87% for Corvi, Ours, Ours-Sync).
- Limited to LDMs whose VAE is available; not applicable to pixel-space diffusion models.
Experimental setup
Datasets: MS COCO LSUN RedCaps LAION-Aesthetics whichfaceisreal WikiArt InstructPix2Pix Nights OpenGL fragment shader images CIFAR
Benchmarks: custom test set of real and LDM-generated images (SD, MJ, Kandinsky, Playground, PixelArt-alpha, LCM), with and without
Metrics: accuracy average precision TPR@5FPR
Baselines: AEROBLADE Ojha-ProGAN Ojha-LDM Cozzolino-LDM Corvi DRCT
Builds on: Corvi et al. detector training recipe CLIP linear probing (UnivFD) AEROBLADE latent diffusion models DDIM inversion
Closest papers by meaning
- FakeInversion: Learning to Detect Images from Unseen Text-to-Image Models by Inverting Stable Diffusion (CVPR 2024 · similarity 0.89)
- When Model Knowledge meets Diffusion Model: Diffusion-assisted Data-free Image Synthesis with Alignment of Domain and Class (ICML 2025 · similarity 0.88)
- AEROBLADE: Training-Free Detection of Latent Diffusion Images Using Autoencoder Reconstruction Error (CVPR 2024 · similarity 0.88)
- LaRE^2: Latent Reconstruction Error Based Method for Diffusion-Generated Image Detection (CVPR 2024 · similarity 0.88)
- Community Forensics: Using Thousands of Generators to Train Fake Image Detectors (CVPR 2025 · similarity 0.88)
- Real-Fake: Effective Training Data Synthesis Through Distribution Matching (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.