Research map / Image generation / Autoregressive and latent generation
OpenSDI: Spotting Diffusion-Generated Images in the Open World
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces OpenSDID, a 300K-image benchmark of VLM-prompted, multi-diffusion-model global and local edits, and MaskCLIP, a CLIP+MAE model built with a Synergizing Pretrained Models (SPM) scheme for jointly detecting and localizing diffusion-generated images.
Problem
Detecting and localizing diffusion-generated images in open-world settings, where user prompts vary, diffusion models keep evolving, and manipulations range from global synthesis to local edits; existing datasets and detectors do not cover all three.
Approach
Combine frozen CLIP and MAE foundation models through learnable prompts (prompt tuning) and cross-attention blocks (VCA, TVCA, VSA) so one model yields both image-level detection and pixel-level localization with better cross-generator generalization.
Key results
- MaskCLIP reaches average localization IoU 0.4271 and F1 0.4941 on OpenSDID, 14.23% and 14.11% relative improvements over the second-best method.
- MaskCLIP achieves average image-level F1 0.7779 and accuracy 0.8198 across generators, 2.38% F1 and 2.05% accuracy above the second-best.
- On the unseen Flux.1 subset, MaskCLIP improves localization IoU by 44.69% over the previous best per the paper's reported relative gains.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MaskCLIP instead of IML-ViT | OpenSDID localization · OpenSDID AVG | IoU: 0.3251 -> 0.4271 | Helped |
| Use MaskCLIP instead of IML-ViT | OpenSDID localization · OpenSDID AVG | F1: 0.3730 -> 0.4941 | Helped |
| Use MaskCLIP instead of RINE | OpenSDID detection · OpenSDID AVG | F1: 0.7598 -> 0.7779 | Helped |
| Use MaskCLIP instead of RINE | OpenSDID detection · OpenSDID AVG | Accuracy: 0.8033 -> 0.8198 | Helped |
| Add prompt tuning to the baseline (ablation) | MaskCLIP · OpenSDID SD1.5 | Image F1: 0.7613 -> 0.8576 | Helped |
| Add VSA after prompt tuning (ablation) | MaskCLIP · OpenSDID SD1.5 | Image F1: 0.8576 -> 0.9083 | Helped |
| Add VSA after prompt tuning (ablation) | MaskCLIP · OpenSDID SD1.5 | Pixel F1: 0.4485 -> 0.4259 | Hurt |
| Add VCA after VSA (ablation) | MaskCLIP · OpenSDID SD1.5 | Pixel F1: 0.4259 -> 0.6532 | Helped |
| Add VCA after VSA (ablation) | MaskCLIP · OpenSDID SD1.5 | Image F1: 0.9083 -> 0.9343 | Helped |
| Add TVCA after VCA (ablation) | MaskCLIP · OpenSDID SD1.5 | Pixel F1: 0.6532 -> 0.7563 | Helped |
| Add TVCA after VCA (ablation) | MaskCLIP · OpenSDID SD1.5 | Image F1: 0.9343 -> 0.9264 | Hurt |
Surprising findings
- NPR achieves better image-level performance than MaskCLIP on the Flux.1 subset, per the paper's own note.
Limitations
- Training uses only SD1.5 fakes, so cross-generator results depend on generalization to unseen models.
- Performance on the Flux.1 subset remains low (IoU 0.1622 average-level localization), indicating limited generalization to newest generators.
Experimental setup
Datasets: Megalith-10M OpenSDID
Benchmarks: OpenSDID
Metrics: IoU F1 accuracy
Baselines: MVSS-Net CAT-Net PSCC-Net ObjectFormer TruFor DeCLIP IML-ViT CNNDet UniFD NPR GramNet FreqNet
Builds on: CLIP Masked Autoencoder (MAE) UniFD DeCLIP IML-ViT TruFor
Closest papers by meaning
- Erase Diffusion: Empowering Object Removal Through Calibrating Diffusion Pathways (CVPR 2025 · similarity 0.83)
- DiffEditor: Boosting Accuracy and Flexibility on Diffusion-based Image Editing (CVPR 2024 · similarity 0.82)
- Diffusing Differentiable Representations (NeurIPS 2024 · similarity 0.81)
- Image Copy Detection for Diffusion Models (NeurIPS 2024 · similarity 0.81)
- RAD: Region-Aware Diffusion Models for Image Inpainting (CVPR 2025 · similarity 0.81)
- Ambient Diffusion Omni: Training Good Models with Bad Data (NeurIPS 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.