Research map / Detection and segmentation / Open-set and few-shot detection
SAFIRE: Segment Any Forged Image Region
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SAFIRE, a SAM-based point-prompted image forgery localization method that segments the source region containing each prompted point, enabling multi-source partitioning while also improving binary forgery localization.
Problem
Image forgery localization is usually framed as binary segmentation of original vs. manipulated pixels, which cannot separate multiple originating sources in an image forged two or more times.
Approach
Use SAM-style point prompts to segment the source region containing each point, with region-to-region contrastive pretraining of the encoder, area-adaptive loss on point masks derived from binary labels, and grid-point inference followed by clustering of representative features to produce multi-source partitions.
Key results
- On binary IFL, SAFIRE reaches 62.6 average F1 fixed and 73.0 average F1 best, ranking first on average and in four of five datasets (TruFor 56.3 / 68.7).
- On SafireMS-Expert 2-source partitioning, SAFIRE (k-means) achieves 90.3 p_mIoU and 80.7 ARI, versus 82.5 p_mIoU for TruFor; baselines cannot produce 3+ source partitions.
- Ablation: replacing point prompting with binary segmentation drops F1 fixed from 62.6 to 28.0.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace region-to-region contrastive pretraining with SAM pretraining (ablation) | SAFIRE · Binary IFL test datasets | F1 fixed: 62.6 -> 48.6 | Hurt |
| Remove area adaptation from AASS loss (ablation) | SAFIRE · Binary IFL test datasets | F1 fixed: 62.6 -> 56.9 | Hurt |
| Replace point prompting with binary segmentation (ablation) | SAFIRE · Binary IFL test datasets | F1 fixed: 62.6 -> 28.0 | Hurt |
| Replace the learned confidence score with a random mask and omit confidence loss (ablation) | SAFIRE · Binary IFL test datasets | F1 fixed: 62.6 -> 43.5 | Hurt |
| Remove all four core components (ablation) | SAFIRE · Binary IFL test datasets | F1 fixed: 62.6 -> 22.3 | Hurt |
| Use point-prompt source partitioning rather than binary segmentation (ablation) | SAFIRE · Binary IFL test datasets | F1 fixed: 28.0 -> 62.6 | Helped |
| Use k-means rather than DBSCAN clustering | SAFIRE · SafireMS-Expert | p_mIoU: 90.3 vs 89.7; 62.2 vs 57.6; 54.0 vs 48.9 | Mixed |
| Use DBSCAN rather than k-means clustering | SAFIRE · SafireMS-Expert | ARI: 80.2 vs 80.7; 60.2 vs 57.8; 59.4 vs 55.2 | Mixed |
Surprising findings
- Point-prompted source partitioning trained only on binary labels substantially outperforms binary segmentation with the same architecture (62.6 vs 28.0 F1 fixed).
- Plain SAM trained on the same data scores only 22.3 average F1 fixed, near the baseline.
Limitations
- SafireMS-Expert is small (238 image-mask pairs), limiting multi-source evaluation scale.
- NC16 images were downscaled for some comparison methods due to memory constraints.
- Number of sources must be fixed (k-means) or estimated by a clustering algorithm (DBSCAN), with DBSCAN performing worse at 2-source ARI-lite comparisons in some metrics.
Experimental setup
Datasets: SafireMS-Auto SafireMS-Expert Columbia COVERAGE CocoGlide RealisticTampering NC16 COCO 2017 DPReview
Benchmarks: Columbia COVERAGE CocoGlide RealisticTampering NC16 SafireMS-Expert
Metrics: permuted F1 fixed permuted F1 best p_mIoU ARI
Baselines: EXIF-SC ManTra-Net SPAN AdaCFA CAT-Net v2 IF-OSN MVSS-Net PSCC-Net TruFor NCL SAM
Builds on: Segment Anything Model InfoNCE CAT-Net setting
Closest papers by meaning
- Learning Discriminative Noise Guidance for Image Forgery Detection and Localization (AAAI 2024 · similarity 0.84)
- CatmullRom Splines-Based Regression for Image Forgery Localization (AAAI 2024 · similarity 0.84)
- MUN: Image Forgery Localization Based on M³ Encoder and UN Decoder (AAAI 2025 · similarity 0.83)
- Reinforced Multi-teacher Knowledge Distillation for Efficient General Image Forgery Detection and Localization (AAAI 2025 · similarity 0.83)
- Toward Real-world Text Image Forgery Localization: Structured and Interpretable Data Synthesis (NeurIPS 2025 · similarity 0.83)
- A Lottery Ticket Hypothesis Approach with Sparse Fine-tuning and MAE for Image Forgery Detection and Localization (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.