Research map / Image generation / Autoregressive and latent generation
CO-SPY: Combining Semantic and Pixel Features to Detect Synthetic Images by AI
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CO-SPY, a synthetic image detector that combines CLIP-based semantic features (with feature-interpolation augmentation) and VAE-reconstruction artifact features via adaptive fusion, and releases CO-SPYBENCH with 22 generators and 50k in-the-wild images.
Problem
Detecting AI-generated images generalizes poorly across newer generative models, unseen content, and lossy JPEG compression; existing semantic and artifact detectors each fail in different scenarios.
Approach
Enhance both feature types first (CLIP ViT-SO400M semantic features with feature-space interpolation; VAE reconstruction-difference artifacts with a ResNet-50 encoder) and then fuse them with a learned adaptive regulator and dropout, rather than naively concatenating them.
Key results
- CO-SPY reaches 96.02% average AP and 87.06% accuracy on CO-SPYBENCH when trained on DRCT, about 7% AP and 11% accuracy above the best baseline.
- On CO-SPYBENCH/in-the-wild, CO-SPY achieves 77.41% average accuracy, about 14% higher than the best baseline.
- The VAE-based artifact detector improves average accuracy by 17-37% over LNP and NPR in the Figure 5 comparison.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine enhanced semantic and artifact features with adaptive fusion | CO-SPY · CO-SPYBENCH | accuracy: 11% | Helped |
| Use VAE reconstruction artifacts instead of low-level upsampling artifacts | Artifact detector · CO-SPYBENCH with JPEG compression | accuracy: 17-37% over LNP and NPR; 28% over DIRE | Helped |
| Use CLIP features and feature-space interpolation for semantic detection | Semantic detector · Synthetic image detection | accuracy: around 10% | Helped |
| Use adaptive fusion rather than the enhanced artifact or semantic detector alone | CO-SPY · Synthetic image detection | accuracy: 3%; 7%; 11% | Helped |
| Apply JPEG compression to artifact-detector inputs | LNP and NPR · CO-SPYBENCH | accuracy: approximately 17% | Hurt |
| Apply JPEG compression to semantic-detector inputs | Fusing and DRCT · CO-SPYBENCH | accuracy: approximately 2% | Hurt |
| Use adaptive fusion instead of straightforward feature-integration methods (ablation) | CO-SPY · Synthetic image detection | detection performance | Helped |
Surprising findings
- Naive integration of semantic and artifact detectors yields only additive gains rather than synergy, according to the authors' evaluation in Appendix E.
- Artifact detectors (LNP, NPR) lose about 17% accuracy under slight JPEG compression, while semantic detectors lose only about 2%.
Limitations
- Semantic detectors trained on DRCT overfit to SD-v1.4 style, so CO-SPY is slightly worse than Fusing on older models such as SD-v1.4 and SD-v1.5.
- Performance decreases under high-strength blurring and noise, though accuracy stays above 75% in extreme cases.
- In-the-wild accuracy (77.41%) remains well below in-distribution accuracy.
Experimental setup
Datasets: DRCT-2M CNNDet/ProGAN training set AIGCDetectBenchmark GenImage CO-SPYBENCH CO-SPYBENCH/in-the-wild
Benchmarks: CO-SPYBENCH CO-SPYBENCH/in-the-wild AIGCDetectBenchmark GenImage
Metrics: average precision accuracy F1 ROC-AUC TPR at low FPR
Baselines: CNNDet FreqFD Fusing LNP UnivFD DIRE FreqNet NPR DRCT AIDE
Builds on: UnivFD DRCT NPR LNP
Closest papers by meaning
- Is Artificial Intelligence Generated Image Detection a Solved Problem? (NeurIPS 2025 · similarity 0.89)
- Community Forensics: Using Thousands of Generators to Train Fake Image Detectors (CVPR 2025 · similarity 0.88)
- Breaking Semantic Artifacts for Generalized AI-generated Image Detection (NeurIPS 2024 · similarity 0.88)
- Stay-Positive: A Case for Ignoring Real Image Features in Fake Image Detection (ICML 2025 · similarity 0.88)
- Dual Data Alignment Makes AI-Generated Image Detector Easier Generalizable (NeurIPS 2025 · similarity 0.88)
- A Sanity Check for AI-generated Image Detection (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.