Research map / Image generation / Autoregressive and latent generation
DefectFill: Realistic Defect Generation with Inpainting Diffusion Model for Visual Inspection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Fine-tunes Stable Diffusion 2 inpainting with LoRA using defect, object, and attention losses plus Low-Fidelity Selection to generate realistic defect images from few references, improving MVTec AD classification and localization.
Problem
Visual inspection models suffer from scarce defect data, and existing defect image generators produce unrealistic defects that limit downstream inspection accuracy.
Approach
Adapts an inpainting diffusion model with LoRA using three custom losses (defect loss on masked region, object loss with random-box masks and "A [Object] with [V*]" prompt, and cross-attention loss aligning the defect token to the mask), then selects the least-reconstructed sample among eight via LPIPS in the masked region.
Key results
- Achieves best KID on 9 of 10 MVTec objects versus DFMGAN and AnomalyDiffusion (e.g., hazelnut KID 1.13 vs 21.16 for DFMGAN).
- Classification accuracy with ResNet-34 improves across all objects, e.g., pill 64.58% to 97.53% and capsule 66.67% to 87.50% over AnomalyDiffusion.
- Localization with UNet achieves best AUROC/AP/F1-max/PRO on all objects; capsule AP improves from 0.57 to 0.75 over AnomalyDiffusion.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DefectFill-generated defects instead of AnoDiff-generated defects for classification training | ResNet-34 · MVTec AD capsule | classification accuracy: 66.67 → 87.50 | Helped |
| Use DefectFill-generated defects instead of AnoDiff-generated defects for classification training | ResNet-34 · MVTec AD pill | classification accuracy: 64.58 → 97.53 | Helped |
| Use DefectFill instead of the strongest baseline for defect generation | DefectFill · MVTec AD carpet | KID: 25.14 → 50.37 | Hurt |
| Add Low-Fidelity Selection (ablation) | DefectFill · MVTec AD generated defects | KID: 28.485 → 25.947 | Helped |
| Add Low-Fidelity Selection (ablation) | DefectFill · MVTec AD generated defects | IC-LPIPS: 0.247 → 0.250 | Helped |
| Add defect loss (ablation) | DefectFill · MVTec AD defect generation | defect fidelity | Helped |
| Add object loss (ablation) | DefectFill · MVTec AD defect generation | semantic alignment | Helped |
| Add attention loss (ablation) | DefectFill · MVTec AD defect generation | defect fidelity | Helped |
| Use an inpainting diffusion model instead of vanilla Stable Diffusion (ablation) | DefectFill with the proposed loss · MVTec AD defect generation | defect realism | Helped |
| Replace the proposed loss with CLiC loss (ablation) | Stable Diffusion 2 defect generation · MVTec AD defect generation | defect realism | Hurt |
Surprising findings
- Baseline CLiC-style loss on vanilla Stable Diffusion tends to reconstruct rather than create holes, showing the inpainting backbone is needed.
Limitations
- Less effective for global structural defects such as misalignment because the approach is local inpainting-based.
- Quantitative evaluation is restricted to 10 MVTec AD objects.
- Leather IC-LPIPS is lower than baselines, which the authors attribute to low-quality diverse samples from baselines.
Experimental setup
Datasets: MVTec AD
Benchmarks: MVTec AD
Metrics: KID IC-LPIPS classification accuracy AUROC AP F1-max PRO
Baselines: DFMGAN AnomalyDiffusion CLiC
Builds on: Latent Diffusion Models Stable Diffusion inpainting CLiC LoRA AnomalyDiffusion
Closest papers by meaning
- Dual-Interrelated Diffusion Model for Few-Shot Anomaly Image Generation (CVPR 2025 · similarity 0.85)
- Exploring the Relationship Between Samples and Masks for Robust Defect Localization (AAAI 2025 · similarity 0.85)
- AnomalyDiffusion: Few-Shot Anomaly Image Generation with Diffusion Model (AAAI 2024 · similarity 0.84)
- Wavelet and Prototype Augmented Query-based Transformer for Pixel-level Surface Defect Detection (CVPR 2025 · similarity 0.84)
- MVREC: A General Few-shot Defect Classification Model Using Multi-View Region-Context (AAAI 2025 · similarity 0.84)
- Towards Open-Vocabulary Industrial Defect Understanding with a Large-Scale Multimodal Dataset (CVPR 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.