Research map / Image generation / Text-to-image diffusion
FakeInversion: Learning to Detect Images from Unseen Text-to-Image Models by Inverting Stable Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FakeInversion, a ResNet50 synthetic-image detector that takes the image plus its Stable Diffusion DDIM inversion noise map and reconstruction, trained on SD fakes vs LAION reals, and SynRIS, a reverse-image-search evaluation benchmark for unseen text-to-image generators.
Problem
Detecting fake images from new high-fidelity text-to-image generators, which existing image-space detectors fail to generalize to; current evaluations also suffer from thematic and stylistic bias between real and fake sets.
Approach
Use features from DDIM inversion and reconstruction through an open-source Stable Diffusion model (noise map, reconstruction, and image together) as detector inputs, so the detector generalizes to unseen generators; pair this with a reverse-image-search real set to build style- and theme-aligned evaluation.
Key results
- FakeInversion trained on SD+LAION reaches 0.759 average AUCROC across evaluation sets versus 0.661 for the best SD-trained UFD retrain and 0.798 average on ProGAN+LSUN training, outperforming prior methods on closed and open generators.
- On the DALL·E 3 subset with SD+LAION training, FakeInversion achieves 0.759 AUCROC compared with 0.480 for retrained UFD.
- LAION-based evaluation underestimates UFD's false positive rate on closed-source generators; RIS-based real sets yield FPR and FID/KID values closer to WebLI-based evaluation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FakeInversion instead of the strongest listed baseline | detectors trained on Stable Diffusion + LAION · SynRIS average | AUCROC: 0.661 -> 0.759 | Helped |
| Use FakeInversion instead of the strongest listed baseline | detectors trained on ProGAN + LSUN · SynRIS average | AUCROC: 0.623 -> 0.798 | Helped |
| Add the input image, DDIM inversion map, and reconstruction rather than RGB alone (ablation) | FakeInversion detector trained on ProGAN + LSUN · unseen text-to-image generators | AUCROC: 0.410 -> 0.854 (DALL·E 2); 0.399 -> 0.642 (DALL·E 3) | Helped |
| Add the input image, DDIM inversion map, and reconstruction rather than RGB alone (ablation) | FakeInversion detector trained on Stable Diffusion + LAION · unseen text-to-image generators | AUCROC: 0.592 -> 0.747 (DALL·E 2); 0.676 -> 0.759 (DALL·E 3) | Helped |
| Add the input image, DDIM inversion map, and reconstruction rather than DDIM residuals alone (ablation) | FakeInversion detector trained on ProGAN + LSUN · unseen text-to-image generators | AUCROC: 0.650 -> 0.854 (DALL·E 2); 0.672 -> 0.642 (DALL·E 3) | Mixed |
| Add the input image, DDIM inversion map, and reconstruction rather than DDIM residuals alone (ablation) | FakeInversion detector trained on Stable Diffusion + LAION · unseen text-to-image generators | AUCROC: 0.650 -> 0.747 (DALL·E 2); 0.672 -> 0.759 (DALL·E 3) | Mixed |
| Use reverse-image-search real images instead of LAION real images for evaluation | UFD · DALL·E 3 | FPR@0.8 recall: 2.6 -> 0.4 | Hurt |
| Use reverse-image-search real images instead of LAION real images for evaluation | UFD · DALL·E 2 | FPR@0.8 recall: 2.7 -> 0.3 | Hurt |
Surprising findings
- Recent detectors (UFD, DMDet) that do well on prior academic benchmarks degrade sharply on the RIS-based benchmark, suggesting overfitting to natural-image content and style distributions.
- The DIRE official checkpoint performs much lower than reported in its paper due to an issue with the released checkpoint.
Limitations
- Detection performance on the new SynRIS benchmark remains far from saturated.
- Inversion requires running Stable Diffusion's VAE and DDIM inversion per image, adding inference cost; classifier-free guidance is not used during inversion.
Experimental setup
Datasets: DiffusionDB LAION ProGAN+LSUN DALL·E 2 fakes from CNNDet Midjourney fakes DALL·E 3 fakes
Benchmarks: SynRIS DALL·E 2 (academic, CNNDet) Craiyon (academic) LDM (academic)
Metrics: AUCROC FPR at 80% recall FID KID average precision accuracy
Baselines: CNNDet DMDet UFD DIRE RGB-only ablation residual-only ablation
Builds on: DDIM inversion DIRE UFD CNNDet Latent Diffusion Models
Closest papers by meaning
- Aligned Datasets Improve Detection of Latent Diffusion-Generated Images (ICLR 2025 · similarity 0.89)
- CO-SPY: Combining Semantic and Pixel Features to Detect Synthetic Images by AI (CVPR 2025 · similarity 0.87)
- A Bias-Free Training Paradigm for More General AI-generated Image Detection (CVPR 2025 · similarity 0.87)
- Spot the Fake: Large Multimodal Model-Based Synthetic Image Detection with Artifact Explanation (NeurIPS 2025 · similarity 0.86)
- Few-Shot Learner Generalizes Across AI-Generated Image Detection (ICML 2025 · similarity 0.86)
- LaRE^2: Latent Reconstruction Error Based Method for Diffusion-Generated Image Detection (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.