Research map / Image generation / Autoregressive and latent generation
Where's the Liability in the Generative Era? Recovery-based Black-Box Detection of AI-Generated Content
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a black-box AI-generated image detector that masks part of an image, has a (LoRA-aligned) surrogate diffusion model recover it, and scores the reconstruction quality (PSNR) to separate generated from real images.
Problem
Detecting AI-generated images from closed-source generators accessible only via API, without model weights or large auxiliary real-image datasets, and generalizing across diffusion model variants.
Approach
Generated images are easier for a generative model to recover when corrupted by masking than real images; a surrogate diffusion inpainting model, LoRA-fine-tuned on a small set (<1,000) of target-API samples to match the target's distribution, measures recovery quality via PSNR for detection.
Key results
- Black-box method with Stable Diffusion surrogate reaches about 86.61-86.81% average mAP across eight diffusion variants, outperforming trained deep network (75.51%) and patch classifier (82.30% best).
- LoRA fine-tuning on target samples cuts FPR95 from 47.90% to 23.60% on guided diffusion datasets (AP 86.74 to 92.97).
- Human respondents identified DALL-E 3 images with 72.33% average accuracy in a 100-question study.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use recovery-based detection with a Stable Diffusion surrogate | AI-generated image detection · eight diffusion-model variant datasets | mean AP: 4.31% | Helped |
| Add parameter-efficient fine-tuning of the surrogate on target-model samples (ablation) | Stable Diffusion surrogate · Guided Diffusion | AP: 86.74 -> 92.97 | Helped |
| Add parameter-efficient fine-tuning of the surrogate on target-model samples (ablation) | Stable Diffusion surrogate · Guided Diffusion | FPR: 47.90 -> 23.60 | Helped |
| Use PSNR instead of SSIM as the recovery discrepancy score (ablation) | Stable Diffusion surrogate detection · Guided Diffusion | AUROC: 56.13% -> 94.19% | Helped |
| Use Guided diffusion as the surrogate without fine-tuning (ablation) | recovery-based detection · Guided Diffusion | AP | Helped |
| Use Guided diffusion as the surrogate without fine-tuning (ablation) | recovery-based detection · DALL-E | AP | Hurt |
| Use Guided diffusion as the surrogate without fine-tuning (ablation) | recovery-based detection · Glide | AP | Hurt |
| Use PSNR rather than SSIM, L1, or L2 to score recovery discrepancy (ablation) | recovery-based detection · Guided Diffusion, LDM, Glide, and DALL-E | detection performance | Helped |
| Use a genhalf mask rather than a thick mask (ablation) | recovery-based detection · fake image detection | detection accuracy | Helped |
Surprising findings
- SSIM scoring performs near chance (AUROC about 56%) while PSNR reaches 94%, so the choice of discrepancy metric is critical.
- A surrogate tuned to guided diffusion performs much worse on DALL-E and GLIDE images than on guided images, indicating limited cross-model transfer.
Limitations
- Performance depends on how well the surrogate's distribution matches the target; mismatch (e.g., DALL-E 3) reduces separation.
- Relies on choice of scoring metric and mask type; PSNR chosen empirically.
- Human study and new DALL-E 3 dataset are limited in scale.
Experimental setup
Datasets: LAION ProGAN real/fake data (baselines)
Benchmarks: Guided diffusion LDM GLIDE DALL-E DALL-E 3 fake detection dataset (new)
Metrics: Average Precision (AP) AUROC FPR95 mAP
Baselines: Trained Deep Network (Wang et al.) Patch Classifier Freq-Spec (AutoGAN) Universal fake detector (CLIP ViT)
Builds on: DIRE DDPM LoRA Universal fake detector
Closest papers by meaning
- Community Forensics: Using Thousands of Generators to Train Fake Image Detectors (CVPR 2025 · similarity 0.87)
- A Bias-Free Training Paradigm for More General AI-generated Image Detection (CVPR 2025 · similarity 0.87)
- CO-SPY: Combining Semantic and Pixel Features to Detect Synthetic Images by AI (CVPR 2025 · similarity 0.86)
- Stay-Positive: A Case for Ignoring Real Image Features in Fake Image Detection (ICML 2025 · similarity 0.85)
- WildFake: A Large-Scale and Hierarchical Dataset for AI-Generated Images Detection (AAAI 2025 · similarity 0.85)
- Is Artificial Intelligence Generated Image Detection a Solved Problem? (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.