Research map / Image generation / Autoregressive and latent generation
Transfer Learning of Real Image Features with Soft Contrastive Loss for Fake Image Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Natural Trace Forensics (NTF), which learns self-supervised homogeneous "natural trace" features from real images and transfers them into a soft-contrastive fake-image classifier, and introduces a 12-generator GAN/DM/multi-step dataset for evaluating generalization.
Problem
Detecting fake images from unseen generative models fails because artifact-based detectors overfit to artifacts of training generators, which change as generators evolve.
Approach
Instead of learning real-vs-fake artifact differences, it treats shared statistical traits of real images (natural traces), decoupled by a self-supervised projection, as an extra target; a soft contrastive loss pulls real images toward these traces and pushes fakes away, so the detector decides by proximity to natural traces.
Key results
- NTF reaches 99.9% average AP on GAN-based fakes with 0.2% FPR.
- On DM-based fakes NTF achieves 91.2% average AP and 82.7% ACC, about 6 points AP above the best baseline.
- On Midjourney and Kolors NTF reaches 78.41% and 78.12% accuracy, versus at most 63.02% for baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NTF instead of Grag2021 | GAN-based fake image detection · GAN-based images, average | AP: 98.9 -> 99.9 | Helped |
| Use NTF instead of Grag2021 | GAN-based fake image detection · GAN-based images, average | ACC: 88.5 -> 92.4 | Helped |
| Use NTF instead of Grag2021 | Diffusion-based fake image detection · Diffusion-based images, average | AP: 85.2 -> 91.2 | Helped |
| Use NTF instead of Ojha2023 | Diffusion-based fake image detection · Diffusion-based images, average | ACC: 72.6 -> 82.7 | Helped |
| Use NTF instead of Ojha2023 | Multi-step fake image detection · Multi-step images, average | AP: 87.5 -> 97.3 | Helped |
| Use NTF instead of Ojha2023 | Multi-step fake image detection · Multi-step images, average | ACC: 91.1 -> 88.3 | Hurt |
| Use NTF instead of Wang2020 | Midjourney detection · Midjourney | accuracy: 63.02 -> 78.41 | Helped |
| Use NTF instead of Grag2021 | Kolors detection · Kolors | accuracy: 60.24 -> 78.12 | Helped |
| Add soft orthogonality loss to natural-trace representation learning (ablation) | NTF · Unseen generative models | generalization performance | Helped |
| Add heterogeneous-feature loss to natural-trace representation learning (ablation) | NTF · Unseen generative models | generalization performance | Helped |
| Add soft contrastive learning (ablation) | NTF · Unseen generative models | generalization performance | Helped |
| Replace ProGAN training data with LDM data (ablation) | NTF · Unseen generative models | generalization performance | Mixed |
Surprising findings
- Ojha2023 (CLIP ViT-L/14 features) slightly beats NTF on multi-step ACC (91.1% vs NTF's lower ACC), which the authors attribute to the CLIP backbone.
- NTF shows a notably higher FNR (around 15%) across most generators, which the authors attribute to limited real-world coverage in training data.
Limitations
- High false negative rate (about 15%) on most generators.
- Substantial degradation under image scaling operations, similar to baselines.
- Mixed results: slightly inferior to some baselines on certain DM models and multi-step ACC.
Experimental setup
Datasets: ProGAN/LSUN dataset (720K train, 4K val) FaceForensics++ LAION self-built GAN/DM/multi-step dataset
Benchmarks: self-built GAN/DM/multi-step dataset Midjourney and Kolors commercial generations
Metrics: AP ACC FPR FNR
Baselines: Xception Wang2020 Grag2021 Ojha2023 SBIs CADDM
Builds on: supervised contrastive learning CNNDetection (Wang2020) UnivFD (Ojha2023) SBIs CADDM
Closest papers by meaning
- Stay-Positive: A Case for Ignoring Real Image Features in Fake Image Detection (ICML 2025 · similarity 0.87)
- Spot the Fake: Large Multimodal Model-Based Synthetic Image Detection with Artifact Explanation (NeurIPS 2025 · similarity 0.86)
- Community Forensics: Using Thousands of Generators to Train Fake Image Detectors (CVPR 2025 · similarity 0.86)
- A Bias-Free Training Paradigm for More General AI-generated Image Detection (CVPR 2025 · similarity 0.86)
- CO-SPY: Combining Semantic and Pixel Features to Detect Synthetic Images by AI (CVPR 2025 · similarity 0.86)
- Real-Fake: Effective Training Data Synthesis Through Distribution Matching (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.