Research map / Vision-language and multimodal / Cross-modal learning
Revisiting Multimodal Fusion for 3D Anomaly Detection from an Architectural Perspective
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Systematically studies how intra- and inter-module multimodal fusion architecture affects RGB+3D anomaly detection and proposes 3D-ADNAS, a DARTS-style NAS over a two-level fusion search space that reports SOTA-level I-AUROC on MVTec 3D-AD and Eyecandies.
Problem
Multimodal fusion architecture (topology) design for 3D anomaly detection (RGB + depth/point cloud) has been largely unexplored, while existing work focuses on fusion strategies rather than architecture.
Approach
Treats the multimodal fusion topology as a searchable two-level space: intra-module (which early/middle/late features and which fusion operations each cell uses) and inter-module (how the early, middle and late modality-specific modules are combined), searched with DARTS-style continuous relaxation, supported by a Dempster-Shafer theoretical analysis of fusion effects.
Key results
- 3D-ADNAS reaches 95.1 mean I-AUROC on MVTec 3D-AD (RGB+3D), above M3DM at 94.5 and EasyNet at 92.6.
- On Eyecandies (RGB+3D), 3D-ADNAS reaches 94.6 mean I-AUROC versus 89.7 for M3DM (per the paper's Table 1 values), and 89.8 mean AUPRO.
- Reported frame rate 24.693 and memory 269.33 on MVTec 3D-AD versus M3DM 0.514 and 6526.12 on a single RTX 4090.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the searched two-level fusion architecture instead of M3DM | 3D-AD on Eyecandies RGB+3D · Eyecandies | I-AUROC (Mean): 4.9% higher | Helped |
| use the searched two-level fusion architecture instead of CFM | 3D-AD on Eyecandies RGB+3D · Eyecandies | AUPRO (Mean): 1.1% higher | Helped |
| use 3D-ADNAS instead of M3DM | 3D-AD on MVTec 3D-AD · MVTec 3D-AD | I-AUROC: 0.6% | Helped |
| use all three MSMs instead of only the early MSM (ablation) | 3D-ADNAS · MVTec 3D-AD | I-AUROC (Mean): +9.8% | Helped |
| use all three MSMs instead of only the middle MSM (ablation) | 3D-ADNAS · MVTec 3D-AD | I-AUROC (Mean): +3.0% | Helped |
| add the middle MSM to the early MSM (ablation) | 3D-ADNAS · MVTec 3D-AD | I-AUROC: +7.7% | Helped |
| remove the late MSM while retaining early and middle MSMs (ablation) | 3D-ADNAS · MVTec 3D-AD | I-AUROC (Mean): 95.1 -> 93.0 | Hurt |
| use only the late MSM rather than all three MSMs (ablation) | 3D-ADNAS · MVTec 3D-AD | I-AUROC (Mean): 95.1 -> 66.4 | Hurt |
Surprising findings
- The late-level fusion module alone degrades 3D-AD performance (mean I-AUROC 66.4), but it consistently helps when combined with other modules.
- Selecting only the first two layers of middle-level features works better than using all middle features.
Limitations
- Search uses basic DARTS gradient descent; the authors note other gradient-based search methods could be applied.
- Few-shot results are only reported at 5/10/50 shots and do not uniformly beat M3DM (50-shot MVTec: M3DM 90.7 vs 89.0).
Experimental setup
Datasets: Eyecandies MVTec 3D-AD
Benchmarks: Eyecandies MVTec 3D-AD
Metrics: I-AUROC P-AUROC AUPRO frame rate memory usage
Baselines: EasyNet M3DM AST CFM BTF Shape_Guided
Builds on: DARTS EasyNet Dempster-Shafer evidence theory
Closest papers by meaning
- Align3D-AD: Cross-Modal Feature Alignment and Dual-Prompt Learning for Zero-shot 3D Anomaly Detection (NeurIPS 2026 · similarity 0.86)
- Multimodal Industrial Anomaly Detection by Crossmodal Feature Mapping (CVPR 2024 · similarity 0.86)
- Real-IAD D3: A Real-World 2D/Pseudo-3D/3D Dataset for Industrial Anomaly Detection (CVPR 2025 · similarity 0.85)
- Fusion or Confusion? Multimodal Complexity Is Not All You Need (NeurIPS 2026 · similarity 0.85)
- Beyond Single-Modal Boundary: Cross-Modal Anomaly Detection through Visual Prototype and Harmonization (CVPR 2025 · similarity 0.84)
- Look Inside for More: Internal Spatial Modality Perception for 3D Anomaly Detection (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.