Research map / Image generation / Autoregressive and latent generation
Community Forensics: Using Thousands of Generators to Train Fake Image Detectors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds Community Forensics, a 2.7M-image dataset sampled from 4803 generators (mostly Hugging Face latent diffusion models plus manually chosen open-source and commercial models), and trains ViT/ConvNeXt fake-image detectors showing that more generator diversity improves out-of-distribution generalization.
Problem
Detectors of AI-generated images generalize poorly to images from generators unseen during training, and existing forensics datasets cover only a handful of generators.
Approach
Scale diversity along the number-of-generators axis by systematically downloading and sampling thousands of community Hugging Face diffusers models, combined with manually selected GANs/diffusion/autoregressive models and commercial models, and train end-to-end CLIP-ViT/ConvNeXt detectors with a modified RandAugment.
Key results
- The best detector (ViT-S, 384 input) reaches 0.987 mAP and 89.2% accuracy on the out-of-distribution comprehensive evaluation set, outperforming prior works by a significant margin.
- Detection performance on out-of-distribution generators (GANs, pixel diffusion) keeps improving as the number of latent diffusion training models grows from 10^1 to 3333, with flattening beyond ~1000 models.
- Adding the manually chosen diverse-architecture set to the systematic set improves results over either set alone, and freezing the pretrained backbone consistently degrades performance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train on more latent diffusion models while fixing the total number of training images (ablation) | Fake image detector · Extended evaluation set across generator types | mAP | Helped |
| Train on 1000 randomly chosen models rather than 10 popular models (ablation) | Fake image detector · Extended evaluation set | mAP | Helped |
| Increase training images from 3K toward 243K (ablation) | Fake image detector · Extended evaluation set | mAP | Helped |
| Add manually selected models with diverse generator architectures to the systematically collected latent diffusion set (ablation) | Fake image detector · Commercial models | mAP | Helped |
| Increase input resolution from 224² to 384² (ablation) | Ours · Synthbuster | mAP: 0.904 -> 0.974 | Helped |
| Freeze the pretrained backbone instead of end-to-end training (ablation) | Fake image detector · Community Forensics evaluation | mAP | Hurt |
| Use a frozen pretrained backbone instead of end-to-end training (ablation) | Fake image detector · Community Forensics evaluation | accuracy | Hurt |
| Use matched rather than mismatched real-image and generator-prompt source datasets (ablation) | Fake image detector · Community Forensics evaluation | mAP | No clear effect |
| Use a ConvNeXt backbone instead of ViT (ablation) | Fake image detector · Community Forensics evaluation | mAP | No clear effect |
| Use a ConvNeXt backbone instead of ViT (ablation) | Fake image detector · Community Forensics evaluation | accuracy | No clear effect |
Surprising findings
- Pairing generated images with semantically aligned real data (source dataset) is not essential; performance differences across pairings were marginal.
- Frozen CLIP backbones, recommended by prior work, consistently perform worse than end-to-end training.
Limitations
- Large portion of the data is Stable Diffusion-based latent diffusion models, limiting architectural diversity.
- Error rates remain too high for critical applications.
- Community-sourced generators may produce inappropriate content requiring scrutiny.
Experimental setup
Datasets: Community Forensics (2.7M generated images, 4803 models) LAION ImageNet COCO FFHQ CelebA MetFaces AFHQ Forchheim IMD2020 Landscapes HQ VISION
Benchmarks: Wang et al. evaluation set Ojha et al. evaluation set Synthbuster GenImage Community Forensics comprehensive evaluation set Community Forensics public evaluation set
Metrics: mAP accuracy
Baselines: Wang et al. [125] Ojha et al. [87] GenImage [133] RED140 [43] Aeroblade LGrad NPR
Builds on: RandAugment CLIP-ViT Wang et al. CNN-generated image detection Ojha et al. UnivFD
Closest papers by meaning
- A Bias-Free Training Paradigm for More General AI-generated Image Detection (CVPR 2025 · similarity 0.92)
- Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentation (NeurIPS 2025 · similarity 0.89)
- WildFake: A Large-Scale and Hierarchical Dataset for AI-Generated Images Detection (AAAI 2025 · similarity 0.89)
- BIAS-ID: A Framework for Analyzing Transformation Biases in AI-Generated Image Detectors (NeurIPS 2026 · similarity 0.89)
- Few-Shot Learner Generalizes Across AI-Generated Image Detection (ICML 2025 · similarity 0.89)
- CO-SPY: Combining Semantic and Pixel Features to Detect Synthetic Images by AI (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.