Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Forensic Self-Descriptions Are All You Need for Zero-Shot Detection, Open-Set Source Attribution, and Clustering of AI-generated Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes forensic self-descriptions: parameters of learned multi-scale linear predictive residual models per image, used for zero-shot AI-image detection, open-set source attribution, and clustering without training on synthetic images.
Problem
Synthetic image detection and source attribution generalize poorly to unseen generators because supervised or embedding-based features are tied to known sources; zero-shot and unsupervised source clustering remain hard.
Approach
Learn a diverse set of linear predictive filters from real images only (energy loss plus spectral diversity regularizer) to extract forensic residuals, then jointly model residuals across scales with per-residual autoregressive filters; the stacked filter parameters form a per-image self-description used with GMMs, K-means, or one-class models.
Key results
- Zero-shot detection averages 0.960 AUC across 96 real-vs-synthetic pairs, versus 0.926 for the next best method (NPR).
- Open-set attribution achieves AU-CRR 0.933 and AU-OSCR 0.913, the highest among compared methods, with known-source accuracy 0.964 (second to Fang et al. at 0.988).
- Clustering with 8 clusters (N=8) reaches 0.78 accuracy, 0.77 purity, and 0.69 NMI, matching the best baseline on accuracy (UFD 0.78) and exceeding it on purity and NMI.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use forensic self-descriptions for zero-shot detection | Forensic self-descriptions · Zero-shot detection, Average | AUC: 0.926 -> 0.960 | Helped |
| use forensic self-descriptions for open-set source attribution | Forensic self-descriptions · Open-set source attribution | AU-CRR: 0.856 -> 0.933 | Helped |
| replace learned diverse residual filters with a 5×5 high-pass filter (ablation) | Forensic self-descriptions · ImageNet-1k vs ProGAN, SDXL, DALLE 3, MJ v6, Firefly | AUC: 0.986 -> 0.913 | Hurt |
| replace learned diverse residual filters with a 3×3 high-pass filter (ablation) | Forensic self-descriptions · ImageNet-1k vs ProGAN, SDXL, DALLE 3, MJ v6, Firefly | AUC: 0.986 -> 0.955 | Hurt |
| replace learned diverse residual filters with Neighbor Pixel Relations (ablation) | Forensic self-descriptions · ImageNet-1k vs ProGAN, SDXL, DALLE 3, MJ v6, Firefly | AUC: 0.986 -> 0.952 | Hurt |
| remove spectral diversity regularization (ablation) | Forensic self-descriptions · ImageNet-1k vs ProGAN, SDXL, DALLE 3, MJ v6, Firefly | AUC: 0.986 -> 0.969 | Hurt |
| remove multi-scale residual modeling (ablation) | Forensic self-descriptions · ImageNet-1k vs ProGAN, SDXL, DALLE 3, MJ v6, Firefly | AUC: 0.986 -> 0.956 | Hurt |
| use 1 learnable filter instead of the full set (ablation) | Forensic self-descriptions · ImageNet-1k vs ProGAN, SDXL, DALLE 3, MJ v6, Firefly | AUC: 0.986 -> 0.951 | Hurt |
| use 4 learnable filters instead of the full set (ablation) | Forensic self-descriptions · ImageNet-1k vs ProGAN, SDXL, DALLE 3, MJ v6, Firefly | AUC: 0.986 -> 0.931 | Hurt |
| use a 7×7 neighborhood for self-descriptions (ablation) | Forensic self-descriptions · ImageNet-1k vs ProGAN, SDXL, DALLE 3, MJ v6, Firefly | AUC: 0.986 -> 0.961 | Hurt |
| use a 5×5 neighborhood for self-descriptions (ablation) | Forensic self-descriptions · ImageNet-1k vs ProGAN, SDXL, DALLE 3, MJ v6, Firefly | AUC: 0.986 -> 0.897 | Hurt |
| use One-Class SVM to utilize self-descriptions (ablation) | Forensic self-descriptions · ImageNet-1k vs ProGAN, SDXL, DALLE 3, MJ v6, Firefly | AUC: 0.986 -> 0.968 | Hurt |
Surprising findings
- A simple 3x3 high-pass filter residual achieves 0.955 AUC, suggesting the learned filters add a moderate gain over basic residuals.
- Zero-shot baselines such as ZED, DE-FAKE, and NPR show large worst-case drops (e.g., NPR 0.239 on Firefly for IN-1k), indicating inconsistent generalization.
Limitations
- Relies on accurate and diverse residuals, which depend on a high-quality, diverse set of real training images.
- Not evaluated on post-processed or social-media-shared images; JPEG robustness only tested on compression quality 50-100.
Experimental setup
Datasets: COCO2017 ImageNet-1k ImageNet-22k MISL Image Database (MIDB) OSSIA DMID Synthbuster
Benchmarks: COCO2017 vs generators ImageNet-1k vs generators ImageNet-22k vs generators MIDB vs generators OSSIA open-set split ImageNet-1k/ProGAN clustering split
Metrics: AUC accuracy AU-CRR AU-OSCR clustering accuracy purity NMI
Baselines: CNNDet PatchFor UFD LGrad DE-FAKE Aeroblade ZED NPR CLIP ResNet-50 DCTCNN RepMix
Builds on: NPR UFD POSE Fang et al. open-set attribution
Closest papers by meaning
- Community Forensics: Using Thousands of Generators to Train Fake Image Detectors (CVPR 2025 · similarity 0.88)
- Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentation (NeurIPS 2025 · similarity 0.87)
- A Bias-Free Training Paradigm for More General AI-generated Image Detection (CVPR 2025 · similarity 0.87)
- ForensicHub: A Unified Benchmark & Codebase for All-Domain Fake Image Detection and Localization (NeurIPS 2025 · similarity 0.85)
- PiD: Generalized AI-Generated Images Detection with Pixelwise Decomposition Residuals (ICML 2025 · similarity 0.85)
- Identity-Aware Vision-Language Model for Explainable Face Forgery Detection (AAAI 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.