Research map / Image generation / Autoregressive and latent generation
Are Images Indistinguishable to Humans Also Indistinguishable to Classifiers?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that neural classifiers easily distinguish real images from diffusion-generated ones (often >98% accuracy) even when FID is low, exposing contradictions with FID and human judgments, and uses the classifier-based distribution classification as a diagnostic tool and for analyzing use of generated data.
Problem
Whether diffusion-generated image distributions are truly close to real ones, since FID and human perception suggest they are; the paper asks how far generated distributions are from real distributions as seen by classifiers.
Approach
Frames distribution classification (real vs. generated, multi-way across generators) as a probe of distributional gap, using standard neural classifiers trained on sufficient samples; compares this with FID and human judgments, and applies it to diagnose diffusion models, explain model autophagy disorder, and guide classifier-guided realism enhancement.
Key results
- Binary classifiers distinguish real from U-ViT-H/2 generated ImageNet-256 images with 99.95% accuracy (ResNet-50), reaching 100% with full ImageNet training set and 1.28M generated images.
- Classifiers struggle to separate EDM2-XS (FID 2.91) from EDM2-XXL (FID 1.81), with ResNet-50 at 74.91% and ConvNeXt-T/ViT-S near chance (57-60%).
- Augmenting CIFAR-10 with EDM-generated data improves supervised accuracy, while replacing real data degrades it; with FlexMatch/FreeMatch, replacement reduces performance while augmentation (via DPT) improves it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train a classifier to distinguish real images from diffusion-generated images | Label-to-image distributions on CIFAR-10 and ImageNet · CIFAR-10 and ImageNet | classification accuracy: most achieving over 98% accuracy | Helped |
| Use 50k samples per distribution instead of fewer training samples | U-ViT-H/2 vs. ImageNet-256 distribution classification · ImageNet-256 | accuracy: over 99.5% accuracy | Helped |
| Increase text-to-image training samples from 5k to 10k | Four-way COCO/Pixart-α/SDXL/Playground-v2.5 classification · COCO2014 text-to-image | accuracy: 88.28 -> 93.45; 76.33 -> 83.53; 80.53 -> 85.13 | Helped |
| Classify distributions within the same EDM2 family at different scales | EDM2-XS and EDM2-XXL · EDM2-XS vs. EDM2-XXL | accuracy: 74.91; 57.35; 59.57 | No clear effect |
| Classify diffusion models with similar FIDs but different architectures | U-ViT-H/2 and DiT-XL/2 · ImageNet-256 | accuracy: 99.66; 97.59; 86.13 | Helped |
| Add further distributions to the classification task | Diffusion-model distribution classification · ImageNet-256 model combinations | accuracy | Helped |
| Use any cropped spatial part of the image for classification (ablation) | U-ViT-H/2 vs. ImageNet-256 · ImageNet-256 | accuracy: 99.96; 99.97; 99.97; 99.78; 99.97; 99.97 | No clear effect |
| Restrict input to minimal low-frequency information (ablation) | U-ViT-H/2 and ImageNet distributions · ImageNet-256 | accuracy | Hurt |
| Replace real CIFAR-10 training data with EDM_S (ablation) | Supervised and semi-supervised CIFAR-10 classification · CIFAR-10 | top-1 accuracy / error rate: 4.97±0.06 -> 5.85±0.02; 4.90±0.04 -> 5.94±0.10 | Hurt |
| Augment real CIFAR-10 with EDM_S instead of replacing real data (ablation) | Supervised and semi-supervised CIFAR-10 classification · CIFAR-10 | top-1 accuracy / error rate: 4.90±0.04 -> 4.68±0.17 | Helped |
| Apply classifier guidance during image generation | ADM · ImageNet image generation | generated images classified as fake (%): 96.19% -> 79.88% | Helped |
Surprising findings
- Classifiers easily separate diffusion models with near-identical FID (U-ViT-H/2, DiT-XL) while humans perform near random guessing, but classifiers struggle with EDM2-XS vs EDM2-XXL where humans perform comparably.
- Adding more distributions to a classification task increased classifier accuracy on prior combinations, contrary to findings in prior dataset-bias work.
Limitations
- Findings are based on specific datasets and models, limiting generalizability.
- Classifiers are trained on real and generated data jointly, unlike universal generated-image detection setups.
- Self-supervised linear classifiers perform worse on CIFAR-10, attributed to pretraining on ImageNet.
Experimental setup
Datasets: CIFAR-10 ImageNet COCO2014
Benchmarks: CIFAR-10 ImageNet COCO2014
Metrics: top-1 accuracy FID human accuracy
Baselines: FID human judgment user study MAE MoCo v3 FlexMatch FreeMatch
Builds on: ADM classifier guidance Name That Dataset dataset bias FID FreeMatch FlexMatch
Closest papers by meaning
- Elucidating the design space of classifier-guided diffusion generation (ICLR 2024 · similarity 0.88)
- Diffusion Classifiers Understand Compositionality, but Conditions Apply (NeurIPS 2025 · similarity 0.87)
- Self-Consuming Generative Models Go MAD (ICLR 2024 · similarity 0.87)
- Trade-Offs in Fine-Tuned Diffusion Models between Accuracy and Interpretability (AAAI 2024 · similarity 0.87)
- Beyond Generation: A Diffusion-based Low-level Feature Extractor for Detecting AI-generated Images (CVPR 2025 · similarity 0.87)
- Balancing Act: Distribution-Guided Debiasing in Diffusion Models (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.