Research map / Image generation / Fast diffusion sampling
InpDiffusion: Image Inpainting Localization via Conditional Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Treats image inpainting localization as conditional mask generation with a diffusion model, adding edge supervision and a dual-stream multi-scale extractor to reduce overconfident predictions and recover subtle boundaries.
Problem
Image inpainting localization (IIL) suffers from overconfident point predictions and difficulty detecting subtle tampering boundaries in inpainted images produced by increasingly realistic GAN and diffusion inpainting.
Approach
Reformulates IIL as conditional denoising of a mask using a diffusion model, conditioned on semantic and edge features from a PVT-based Adaptive Conditional Network; a dual-decoder denoiser predicts both the mask and its edge at each step, with an edge supervision loss that constrains sampling randomness. A Dual-stream Multi-scale Feature Extractor (DMFE) built from dilated asymmetric convolutions
Key results
- On Inpaint32K, InpDiffusion reaches 93.7 / 96.2 / 95.6 / 84.3 AUC across TM, CNN, GAN, DM inpainting types, versus best baseline ECNet at 86.4 / 90.5 / 88.3 / 78.1.
- On IMD in the fine-tuning setting, InpDiffusion achieves 85.5 AUC, 10.2 points above the second-best method (ECNet at 75.3).
- Ablation on Inpaint32K shows removing edge supervision drops TM AUC from 93.7 to 91.6 and removing DMFE drops CNN AUC from 96.2 to 94.0 (with ES) or 93.3 (without ES).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add edge supervision to the denoising network (ablation) | InpDiffusion · Inpaint32K TM | AUC: 91.6 -> 93.7 | Helped |
| Add edge supervision to the denoising network (ablation) | InpDiffusion · Inpaint32K CNN | AUC: 93.3 -> 96.2 | Helped |
| Add edge supervision to the denoising network (ablation) | InpDiffusion · Inpaint32K GAN | AUC: 93.7 -> 95.6 | Helped |
| Add edge supervision to the denoising network (ablation) | InpDiffusion · Inpaint32K DM | AUC: 82.4 -> 84.3 | Helped |
| Add DMFE while retaining edge supervision (ablation) | InpDiffusion · Inpaint32K CNN | AUC: 94.0 -> 96.2 | Helped |
| Add DMFE while retaining edge supervision (ablation) | InpDiffusion · Inpaint32K TM | AUC: 91.9 -> 93.7 | Helped |
| Add DMFE while retaining edge supervision (ablation) | InpDiffusion · Inpaint32K GAN | AUC: 93.5 -> 95.6 | Helped |
| Add DMFE while retaining edge supervision (ablation) | InpDiffusion · Inpaint32K DM | AUC: 82.8 -> 84.3 | Helped |
| Increase sampling steps up to 10 (ablation) | InpDiffusion · Inpaint32K, all inpainting types | AUC | Helped |
| Increase sampling steps beyond 10 (ablation) | InpDiffusion · Inpaint32K, all inpainting types | AUC | No clear effect |
| Set mask-to-edge loss ratio to 7:3 (ablation) | InpDiffusion · Inpaint32K, all inpainting types | AUC | Helped |
| Use SNR Shift -2log(6) instead of -2log(12) (ablation) | InpDiffusion · Training | AUC | Helped |
Surprising findings
- MVSSNet outperforms InpDiffusion on the NS inpainting technique in DID, which the authors attribute to MVSSNet's extra pre-training on a 60K-image dataset.
- The diffusion model's inference time (27.93 ms) is higher than MVSS-Net (18.83 ms) and EC-Net (24.76 ms), though its parameter count is lower than MVSS-Net.
Limitations
- Inference is slower than the compared discriminative baselines due to iterative denoising.
- Underperforms MVSSNet on the NS inpainting technique in DID.
- Baselines were re-trained on Inpaint32K, while MVSSNet's stronger result relies on larger pre-training data, making some comparisons uneven.
Experimental setup
Datasets: Inpaint32K DID AutoSplice IMD Nist
Benchmarks: Inpaint32K DID AutoSplice IMD Nist
Metrics: pixel-level AUC
Baselines: MantraNet MVSSNet TANet MFINet CFLNet ECNet
Builds on: denoising diffusion probabilistic models Pyramid Vision Transformer Res2Net SNR-based variance schedule
Closest papers by meaning
- RAD: Region-Aware Diffusion Models for Image Inpainting (CVPR 2025 · similarity 0.88)
- Image Inpainting via Iteratively Decoupled Probabilistic Modeling (ICLR 2024 · similarity 0.87)
- In-N-Out: Lifting 2D Diffusion Prior for 3D Object Removal via Tuning-Free Latents Alignment (NeurIPS 2024 · similarity 0.87)
- DesignEdit: Unify Spatial-Aware Image Editing via Training-free Inpainting with a Multi-Layered Latent Diffusion Framework (AAAI 2025 · similarity 0.87)
- Image Inpainting via Tractable Steering of Diffusion Models (ICLR 2024 · similarity 0.87)
- Structure Matters: Tackling the Semantic Discrepancy in Diffusion Models for Image Inpainting (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.