Research map / Detection and segmentation / Open-set and few-shot detection
Multimodal Industrial Anomaly Detection by Crossmodal Feature Mapping
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a memory-bank-free multimodal (RGB + point cloud) anomaly detector that learns lightweight MLPs mapping frozen 2D and 3D Transformer features across modalities, with layer pruning for speed and memory gains.
Problem
Unsupervised multimodal RGB-3D industrial anomaly detection and segmentation, where existing memory-bank methods are slow and memory-heavy.
Approach
Learn two crossmodal feature mapping MLPs (2D→3D and 3D→2D) on nominal samples between frozen feature extractors and flag anomalies where both predicted features disagree with observed ones, aggregated by pixel-wise product; no memory bank is needed.
Key results
- On MVTec 3D-AD, the full model reaches mean I-AUROC 0.954 and AUPRO@30% 0.971, outperforming M3DM (0.945 and 0.964).
- Under the stricter AUPRO@1% metric, the method reaches 0.455 vs 0.394 for M3DM.
- Ours-T runs at 42.8 fps versus 0.514 fps for M3DM, with 48 MB memory versus 6526 MB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace memory-bank multimodal detection with crossmodal feature mapping | MVTec 3D-AD · MVTec 3D-AD | I-AUROC: 0.945 -> 0.954 | Helped |
| Replace memory-bank multimodal detection with crossmodal feature mapping | MVTec 3D-AD · MVTec 3D-AD | AUPRO@30%: 0.964 -> 0.971 | Helped |
| Replace memory-bank multimodal detection with crossmodal feature mapping | MVTec 3D-AD · MVTec 3D-AD | AUPRO@1%: 0.394 -> 0.455 | Helped |
| Use crossmodal feature mapping instead of M3DM | Eyecandies · Eyecandies | AUPRO@30%: 0.882 -> 0.887 | Helped |
| Use crossmodal feature mapping instead of M3DM | Eyecandies · Eyecandies | I-AUROC: 0.897 -> 0.881 | Hurt |
| Use crossmodal feature mapping instead of M3DM | Eyecandies · Eyecandies | P-AUROC: 0.977 -> 0.974 | Hurt |
| Add pixel-wise product aggregation of the 2D and 3D anomaly maps (ablation) | Crossmodal feature mapping · MVTec 3D-AD | AUPRO@1%: 0.430 -> 0.455 | Helped |
| Replace pixel-wise product aggregation with pixel-wise maximum (ablation) | Crossmodal feature mapping · MVTec 3D-AD | AUPRO@1%: 0.455 -> 0.400 | Hurt |
| Replace crossmodal mapping with intramodal feature reconstruction (ablation) | Feature mapping framework · MVTec 3D-AD | AUPRO@1%: 0.450 -> 0.426 | Hurt |
| Prune both feature extractors at layer 8 (ablation) | Ours · MVTec 3D-AD | I-AUROC: 0.954 -> 0.960 | Helped |
| Prune both feature extractors at layer 4 (ablation) | Ours · MVTec 3D-AD | I-AUROC: 0.954 -> 0.948 | Hurt |
| Prune both feature extractors at layer 1 (ablation) | Ours · MVTec 3D-AD | I-AUROC: 0.954 -> 0.899 | Hurt |
Surprising findings
- Evaluation at a tighter FPR threshold (AUPRO@1%) shows that current multimodal AD methods perform dramatically worse than under the standard 30% threshold.
Limitations
- Requires both modalities at training and test time, so it cannot be applied to pure 2D or pure 3D AD.
- Layer pruning gives speed and memory gains with some loss in accuracy (e.g., Ours-T I-AUROC 0.899).
Experimental setup
Datasets: MVTec 3D-AD Eyecandies ImageNet ShapeNet
Benchmarks: MVTec 3D-AD Eyecandies
Metrics: I-AUROC P-AUROC AUPRO@30% AUPRO@1% frame rate memory (MB)
Baselines: BTF AST M3DM DepthGAN DepthAE DepthVM VoxelGAN VoxelAE VoxelVM
Builds on: PatchCore M3DM BTF DFR
Closest papers by meaning
- Real-IAD D3: A Real-World 2D/Pseudo-3D/3D Dataset for Industrial Anomaly Detection (CVPR 2025 · similarity 0.90)
- Supervised Anomaly Detection for Complex Industrial Images (CVPR 2024 · similarity 0.87)
- 3CAD: A Large-Scale Real-World 3C Product Dataset for Unsupervised Anomaly Detection (AAAI 2025 · similarity 0.87)
- MMAD: A Comprehensive Benchmark for Multimodal Large Language Models in Industrial Anomaly Detection (ICLR 2025 · similarity 0.87)
- ADPretrain: Advancing Industrial Anomaly Detection via Anomaly Representation Pretraining (NeurIPS 2025 · similarity 0.86)
- InCLAD: A Continual Learning Benchmark for Industrial Visual Anomaly Detection (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.