Research map / Image generation / Autoregressive and latent generation
Is Artificial Intelligence Generated Image Detection a Solved Problem?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces AIGIBench, a 23-subset benchmark that evaluates 11 AI-generated image detectors on generalization, degradation robustness, augmentation, and test-time pre-processing, finding large real-world performance drops.
Problem
Whether AI-generated image (AIGI) detectors that report high accuracy in controlled settings remain reliable under real-world conditions such as unseen generators, social-media content, degradations, augmentation choices, and test-time pre-processing.
Approach
A unified benchmark with four tasks mirroring the real-world AIGI detection pipeline (multi-source generalization, multi-degradation robustness, augmentation sensitivity, pre-processing sensitivity), including newly generated fakes from many GANs/diffusion models, deepfake and personalized-generation methods, plus in-the-wild samples from social media and AI art platforms; separates accuracy into
Key results
- SAFE achieves the highest mean accuracy across 25 test subsets (79.9% under Setting-II; 79.2% under Setting-I per the text).
- Under JPEG compression and Gaussian noise, detectors' F.Acc. collapses toward 0% while R.Acc. stays near 100%, e.g., CNNDetection F.Acc. 0.8% under JPEG in Setting-I.
- Switching test pre-processing from Resize to Crop raises SAFE R.Acc. from 63.3% to 96.8% while F.Acc. shows little or negative gain (63.0% for SAFE after crop, roughly unchanged).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with ProGAN and SD-v1.4 instead of ProGAN alone | AIGI detectors · AIGIBench | R.Acc. | Helped |
| train with ProGAN and SD-v1.4 instead of ProGAN alone | AIGI detectors · AIGIBench | F.Acc. | Hurt |
| add JPEG compression at quality factor 50 to test images | AIGI detectors · AIGIBench Setting-II robustness | F.Acc. | Hurt |
| add Gaussian noise with standard deviation σ = 4 to test images | AIGI detectors · AIGIBench Setting-II robustness | F.Acc. | Hurt |
| add rotation augmentation (ablation) | CLIPDetection · AIGIBench Setting-II | accuracy: 72.5 -> 70.5 | Hurt |
| add rotation augmentation (ablation) | FreqNet · AIGIBench Setting-II | accuracy: 66.2 -> 68.0 | Helped |
| add rotation augmentation (ablation) | NPR · AIGIBench Setting-II | accuracy: 67.9 -> 68.7 | Helped |
| add rotation augmentation (ablation) | DFFreq · AIGIBench Setting-II | accuracy: 71.1 -> 72.7 | Helped |
| add rotation augmentation (ablation) | SAFE · AIGIBench Setting-II | accuracy: 79.5 -> 76.1 | Hurt |
| replace Resize with Crop (ablation) | CLIPDetection · AIGIBench Setting-II | accuracy: 72.5 -> 66.5 | Hurt |
| replace Resize with Crop (ablation) | FreqNet · AIGIBench Setting-II | accuracy: 66.2 -> 74.2 | Helped |
| replace Resize with Crop (ablation) | NPR · AIGIBench Setting-II | accuracy: 67.9 -> 68.2 | No clear effect |
Surprising findings
- Crop pre-processing and common augmentations mainly raise real-image accuracy (R.Acc.) while fake-image accuracy (F.Acc.) stays flat or drops.
- Most detectors maintain high R.Acc. under perturbations yet their F.Acc. falls sharply, so high reported accuracy masks poor detection of fakes.
Limitations
- Only two training settings (ProGAN; ProGAN plus SD-v1.4) are considered.
- Evaluation covers a fixed set of 11 detectors and does not include larger pre-trained training datasets.
- Some benchmark results are described inconsistently across the text, and the Crop vs Resize analysis rests on a single detector comparison.
Experimental setup
Datasets: ProGAN training images SD-v1.4 training images FFHQ CelebA-HQ Open Images V7 SocialRF CommunityAI
Benchmarks: AIGIBench
Metrics: Accuracy average precision R.Acc. F.Acc.
Baselines: ResNet-50 CNNDetection Gram-Net LGrad CLIPDetection FreqNet NPR LaDeDa DFFreq AIDE SAFE
Builds on: CLIPDetection NPR AIDE SAFE FreqNet
Closest papers by meaning
- Are High-Quality AI-Generated Images More Difficult for Models to Detect? (ICML 2025 · similarity 0.89)
- CO-SPY: Combining Semantic and Pixel Features to Detect Synthetic Images by AI (CVPR 2025 · similarity 0.89)
- A Sanity Check for AI-generated Image Detection (ICLR 2025 · similarity 0.88)
- WildFake: A Large-Scale and Hierarchical Dataset for AI-Generated Images Detection (AAAI 2025 · similarity 0.87)
- Few-Shot Learner Generalizes Across AI-Generated Image Detection (ICML 2025 · similarity 0.86)
- Beyond Generation: A Diffusion-based Low-level Feature Extractor for Detecting AI-generated Images (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.