Research map / Image generation / Text-to-image diffusion
Text-Guided Variational Image Generation for Industrial Anomaly Detection and Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a text-guided variational (VQGAN-based) image generator that creates non-defective images from a single or few samples, using WordNet-derived prompts and CLIP-based selection, to augment anomaly detection and segmentation models.
Problem
Industrial anomaly detection suffers from scarce and imbalanced non-defective training data, and naive text-to-image generators produce images that do not help anomaly detectors.
Approach
Extends VQGAN so the decoder samples latents from a predicted per-patch variance, selects the best WordNet-based keyword prompt via CLIP similarity, and picks the generated image set whose CLIP visual features best match the prompt, then uses those images to augment baseline detectors such as PatchCore.
Key results
- Average improvement of 15.7% in one-shot, 14.1% in few-shot, and 5.6% in full-shot detection AUROC across baselines on MVTecAD.
- On PatchCore one-shot hazelnut preliminary test, the method reaches 93.3 detection and 94.6 segmentation vs 88.3 and 94.3 for the original image only.
- Ablation on PatchCore shows the text prompt adds +4.6% image-level AUROC and variance-aware generation adds +4.3% image-level AUROC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add generated non-defective images | Patchcore · MVTecAD hazelnut | detection: 88.3 -> 93.3 | Helped |
| Add generated non-defective images | Patchcore · MVTecAD hazelnut | segmentation: 94.3 -> 94.6 | Helped |
| Use the text prompt instead of removing it (ablation) | Patchcore · Patchcore ablation | Image-Level (%): 77.6(±0.6) -> 79.1(±0.6) | Helped |
| Use the text prompt instead of removing it (ablation) | Patchcore · Patchcore ablation | Pixel-Level (%): 97.1(±0.0) -> 96.9(±0.0) | No clear effect |
| Use variance-aware generation instead of removing variance (ablation) | Patchcore · Patchcore ablation | Image-Level (%): 74.3(±0.4) -> 78.9(±0.5) | Helped |
| Use variance-aware generation instead of removing variance (ablation) | Patchcore · Patchcore ablation | Pixel-Level (%): 97.0(±0.0) -> 97.3(±0.0) | Helped |
| Vary the number of generated images (ablation) | Patchcore · Patchcore ablation | Image-Level (%): 78.6(±0.4); 77.9(±0.4); 78.4(±0.5); 79.0(±0.7); 78.5(±0.3); 78.4(±0.5) | Mixed |
| Vary the number of generated images (ablation) | Patchcore · Patchcore ablation | Pixel-Level (%): 97.0(±0.0); 96.9(±0.0); 97.0(±0.0); 97.0(±0.0); 96.9(±0.1); 96.9(±0.0) | No clear effect |
| Use the proposed method | Four baseline models · MVTecAD one-shot | average improvement: +15.7% | Helped |
| Use the proposed method | Four baseline models · MVTecAD few-shot (5 images) | average improvement: +14.1% | Helped |
| Use the proposed method | Four baseline models · MVTecAD full-shot | average improvement: +5.6% | Helped |
| Use the proposed method | Patchcore · BTAD object type, 1st class | detection AUROC: +20.2% | Helped |
Surprising findings
- Higher image quality from DALL-E did not improve anomaly detection performance; web images performed better than several generative models.
- Adding more generated images does not monotonically improve performance; it saturates and excessive generated non-defective data can act as noise.
Limitations
- Performance gains vary across classes and failures occur in more complex real-world settings.
- Performance is sensitive to the number of generated images and can saturate or degrade with too many.
Experimental setup
Datasets: MVTecAD BTAD MVTec-LOCO AD WordNet
Benchmarks: MVTecAD BTAD MVTec-LOCO AD
Metrics: AUROC Inception Score SSIM PSNR LPIPS
Baselines: PatchCore CFA Reverse distillation WinCLIP DALL-E Midjourney VQGAN-CLIP
Builds on: VQGAN CLIP PatchCore WordNet
Closest papers by meaning
- AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models (AAAI 2024 · similarity 0.87)
- MMAD: A Comprehensive Benchmark for Multimodal Large Language Models in Industrial Anomaly Detection (ICLR 2025 · similarity 0.86)
- Automated Defect Report Generation for Enhanced Industrial Quality Control (AAAI 2024 · similarity 0.85)
- Supervised Anomaly Detection for Complex Industrial Images (CVPR 2024 · similarity 0.85)
- Towards Training-free Anomaly Detection with Vision and Language Foundation Models (CVPR 2025 · similarity 0.85)
- Dual-Interrelated Diffusion Model for Few-Shot Anomaly Image Generation (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.