Research map / Image generation / Autoregressive and latent generation
A Sanity Check for AI-generated Image Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the Chameleon benchmark of human-indistinguishable AI-generated and real images, shows nine off-the-shelf AI-generated image detectors fail on it, and proposes AIDE, a hybrid detector combining CLIP semantics with DCT-selected high/low-frequency patch noise features.
Problem
Existing AI-generated image detectors are trained on GAN images and tested on simple random samples from generators, so it is unclear whether detection is solved for realistic, human-indistinguishable images.
Approach
A "Turing-test" filtered high-resolution benchmark (Chameleon) that keeps only AI images two annotators judged real, plus a mixture-of-experts detector (AIDE) fusing CLIP ConvNeXt semantic embeddings with SRM-filtered noise features from DCT-selected extreme-frequency patches.
Key results
- AIDE reaches 92.77% mean accuracy on AIGCDetectBenchmark, a +3.5 point gain over PatchCraft (89.31%).
- On GenImage, AIDE averages 86.88% accuracy versus 82.30% for PatchCraft (+4.6 points).
- On Chameleon, nearly all nine off-the-shelf detectors and AIDE stay near chance; AIDE reaches 65.77% when trained on all GenImage data, with a high fake-image miss rate (fake accuracy 26.80%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AIDE instead of PatchCraft | AIGCDetectBenchmark · AIGCDetectBenchmark | accuracy (mean, %): 89.31 -> 92.77; +3.5% | Helped |
| Use AIDE instead of PatchCraft | GenImage · GenImage | accuracy (mean, %): 82.30 -> 86.88; 4.6% | Helped |
| Add high-frequency patch features (PFE-H) (ablation) | AIDE · AIGCDetectBenchmark | accuracy (mean, %): 84.20 -> 92.77 | Helped |
| Add low-frequency patch features (PFE-L) (ablation) | AIDE · AIGCDetectBenchmark | accuracy (mean, %): 80.69 -> 92.77 | Helped |
| Add semantic feature extraction (SFE) (ablation) | AIDE · AIGCDetectBenchmark | accuracy (mean, %): 76.70% vs 92.77% | Helped |
| Replace CLIP-ConvNeXt with CLIP-ViT (ablation) | AIDE · AIGCDetectBenchmark | accuracy (mean, %): 92.77% -> 80.87% | Hurt |
| Train AIDE on All GenImage instead of using the listed baseline training setups | AIDE · Chameleon | accuracy (%): 60.89 -> 65.77 | Mixed |
Surprising findings
- Most detectors classify AI-generated Chameleon images as real, with fake-image accuracy often below 10%.
- CLIP ViT backbone gave 80.87% on benchmark 1 versus 92.77% for CLIP ConvNeXt, which the authors attribute to CNN low-level sensitivity.
Limitations
- Performance on Chameleon remains low, so the detection problem is not solved.
- The Chameleon dataset is of limited scale.
- Results are reported without error bars or statistical significance tests.
Experimental setup
Datasets: ProGAN training set (LSUN) SD v1.4 training set (ImageNet) AIGCDetectBenchmark GenImage
Benchmarks: AIGCDetectBenchmark GenImage Chameleon
Metrics: accuracy average precision
Baselines: CNNSpot FreDect Fusing LNP LGrad UnivFD DIRE PatchCraft NPR
Builds on: CNNSpot UnivFD PatchCraft DIRE OpenCLIP
Closest papers by meaning
- Breaking Semantic Artifacts for Generalized AI-generated Image Detection (NeurIPS 2024 · similarity 0.89)
- Are High-Quality AI-Generated Images More Difficult for Models to Detect? (ICML 2025 · similarity 0.88)
- Is Artificial Intelligence Generated Image Detection a Solved Problem? (NeurIPS 2025 · similarity 0.88)
- Beyond Generation: A Diffusion-based Low-level Feature Extractor for Detecting AI-generated Images (CVPR 2025 · similarity 0.88)
- CO-SPY: Combining Semantic and Pixel Features to Detect Synthetic Images by AI (CVPR 2025 · similarity 0.87)
- WildFake: A Large-Scale and Hierarchical Dataset for AI-Generated Images Detection (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.