Research map / Vision-language and multimodal / Multimodal LLMs

Towards Open-Vocabulary Industrial Defect Understanding with a Large-Scale Multimodal Dataset

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces IMDD-1M, a 1.24M-pair industrial defect dataset with text annotations, and a diffusion U-Net trained from scratch on it with mask-based open-vocabulary segmentation/classification heads that claim strong data efficiency with 200 samples per class.

Problem

Industrial defect detection lacks large-scale multimodal (image-text) data, and general VLMs trained on natural images lack industrial visual-semantic knowledge; current defect datasets are small and lack text annotations.

Approach

Train a Stable Diffusion v1.5-style U-Net from random initialization on a million-scale industrial image-text corpus, add an implicit captioner that generates pseudo text embeddings from CLIP image features so captions are not needed downstream, and attach a Mask2Former head for open-vocabulary classification and grounding on frozen diffusion features.

Key results

• Mask-based model with 200 samples per class reaches 74.6% mAP@0.5 on MVTec AD object detection versus 78.3% for YOLOv8-m.
• Segmentation on MVTec AD and VisA averages 91.0% accuracy and 52.9% IoU, with 96.1 P-AUC-ROC and 90.2 AUC-PRO at 200 samples per class versus full-data baselines at about 97-98.
• Classification averages 96.7% accuracy across MVTec AD, VisA, Magnetic Tile and Steel Surface.

Surprising findings

• The paper reports that training from random initialization beats fine-tuning Stable Diffusion weights by 8.2 mIoU, suggesting natural-image priors hinder industrial defect learning.

Limitations

• Requires 576 H100 GPU-hours for stage 1 pretraining and 76GB peak memory per GPU, limiting accessibility.
• Inference at 0.35 s per image and 18.7 GB memory is too slow or heavy for real-time or edge industrial inspection.
• Covers only visible-light 2D RGB imaging; no X-ray, infrared, or 3D data.

Experimental setup

Datasets: IMDD-1M MVTec AD VisA BTAD DAGM KolektorSDD Magnetic Tile NEU-DET

Benchmarks: MVTec AD VisA Magnetic Tile

Metrics: accuracy mAP@0.5 mAP@0.75 IoU F1 Dice P-AUC-ROC AUC-PRO FID IS

Baselines: YOLOv8-m MuSc PromptAD DMAD DDAD SimpleNet FAIR Stable Diffusion XL

Builds on: Stable Diffusion v1.5 Mask2Former CLIP DDPM

Closest papers by meaning

• MMAD: A Comprehensive Benchmark for Multimodal Large Language Models in Industrial Anomaly Detection (ICLR 2025 · similarity 0.89)
• Zero-Shot Learning in Industrial Scenarios: New Large-Scale Benchmark, Challenges and Baseline (AAAI 2025 · similarity 0.88)
• MVREC: A General Few-shot Defect Classification Model Using Multi-View Region-Context (AAAI 2025 · similarity 0.87)
• Automated Defect Report Generation for Enhanced Industrial Quality Control (AAAI 2024 · similarity 0.87)
• Multimodal Industrial Anomaly Detection by Crossmodal Feature Mapping (CVPR 2024 · similarity 0.85)
• AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models (AAAI 2024 · similarity 0.85)