Research map / Vision-language and multimodal / CLIP and zero-shot
Zero-Shot Learning in Industrial Scenarios: New Large-Scale Benchmark, Challenges and Baseline
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds MMIO, an 80K-sample multi-scene industrial defect dataset, and proposes RTVP, a Mobile-SAM-based zero-shot detector with expert-guided domain adaptation and text-visual prompt interaction.
Problem
Applying large visual-language models and SAM-style prompting to industrial defect detection is hard: user-provided prompts are subjective and noisy, and there is little large-scale multi-scene industrial data for zero-shot learning.
Approach
Uses a CNN domain expert to supervise domain adaptation of Mobile-SAM, generates refined visual prompts automatically via channel/spatial activation and sparse top-k pixel selection, and fuses them with text embeddings through max-sigmoid attention, removing the need for user-provided per-image prompts.
Key results
- On MMIO zero-shot, RTVP reaches 24.7 AP and 35.7 AP50, versus 22.9 AP for DetCLIP-T and 20.2 AP for Grounding DINO-T.
- On MMIO closed scenarios, RTVP reaches 42.4 AP, above YOLOv10-S (41.7) and YOLOv9 (41.1).
- Ablation removing the expert domain adapter drops zero-shot AP from 24.7 to 18.5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the expert domain adapter (ablation) | RTVP · MMIO closed | AP: 30.2 -> 42.4 | Helped |
| Add the refined visual prompt (ablation) | RTVP · MMIO closed | AP: 38.6 -> 42.4 | Helped |
| Add text-visual interaction (ablation) | RTVP · MMIO closed | AP: 40.0 -> 42.4 | Helped |
| Add the expert domain adapter (ablation) | RTVP · MMIO zero-shot | AP: 18.5 -> 24.7 | Helped |
| Add the refined visual prompt (ablation) | RTVP · MMIO zero-shot | AP: 20.0 -> 24.7 | Helped |
| Add text-visual interaction (ablation) | RTVP · MMIO zero-shot | AP: 22.8 -> 24.7 | Helped |
| Use patch size 32 rather than 64 (ablation) | RTVP sparse modelling · MMIO zero-shot | AP: 21.3 -> 24.7 | Helped |
| Use patch size 32 rather than 128 (ablation) | RTVP sparse modelling · MMIO zero-shot | AP: 19.9 -> 24.7 | Helped |
| Use RTVP instead of YOLOv10-S | MMIO closed detection · MMIO closed | AP: 41.7 -> 42.4 | Helped |
| Use RTVP instead of DetCLIP-T | MMIO zero-shot detection · MMIO zero-shot | AP: 22.9 -> 24.7 | Helped |
| Use RTVP instead of YOLOv8-S | COCO closed-scene detection · MS-COCO | AP: 44.9 -> 47.2 | Helped |
| Use RTVP instead of YOLO-W with O365 pretraining | LVIS zero-shot detection · LVIS zero-shot (O365) | AP: 23.5 -> 24.1 | Helped |
Surprising findings
- Absolute AP on MMIO zero-shot is very low across all methods, which the authors attribute to the task's difficulty.
Limitations
- Absolute zero-shot AP remains low (24.7), so the benchmark is far from solved.
- The method relies on a CNN expert trained on visible classes, adding a dependency on supervised industrial data.
Experimental setup
Datasets: MMIO-80K MS-COCO LVIS Objects365 GoldG
Benchmarks: MMIO-80K closed MMIO-80K zero-shot MS-COCO LVIS
Metrics: AP AP50 AP75 Precision Recall
Baselines: YOLOv8 YOLOv9 YOLOv10 LiteYOLO-ID LF-YOLO Steel det MDETR GLIP-T GLIPv2-T DetCLIP-T YOLO-World-S Grounding DINO-T
Builds on: Segment Anything (SAM) Mobile-SAM YOLO-World Fine-Grained Visual Prompting (FGVP) CLIP
Closest papers by meaning
- Towards Open-Vocabulary Industrial Defect Understanding with a Large-Scale Multimodal Dataset (CVPR 2026 · similarity 0.88)
- MMAD: A Comprehensive Benchmark for Multimodal Large Language Models in Industrial Anomaly Detection (ICLR 2025 · similarity 0.88)
- AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models (AAAI 2024 · similarity 0.88)
- Explanatory Instructions: Towards Unified Vision Tasks Understanding and Zero-shot Generalization (ICML 2025 · similarity 0.85)
- MVREC: A General Few-shot Defect Classification Model Using Multi-View Region-Context (AAAI 2025 · similarity 0.85)
- On the Test-Time Zero-Shot Generalization of Vision-Language Models: Do We Really Need Prompt Learning? (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.