Research map / Label-efficient and robust learning / OOD and anomaly detection
MultiOOD: Scaling Out-of-Distribution Detection for Multiple Modalities
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MultiOOD, a multimodal (video, optical flow, audio) OOD detection benchmark built from five action recognition datasets, plus the A2D modality-discrepancy training loss and NP-Mix outlier synthesis that improve existing unimodal OOD scores.
Problem
Out-of-distribution detection has been studied almost only for unimodal image inputs, and no benchmark existed for multimodal OOD detection across varied dataset sizes and modality combinations.
Approach
Observes that softmax predictions from different modalities agree on ID data but diverge on OOD data (Modality Prediction Discrepancy); A2D trains modalities to agree on the ground-truth class and disagree on the remaining classes via a Hellinger-distance loss, and NP-Mix synthesizes outliers by mixing prototypes of nearest-neighbor classes.
Key results
- A2D+NP-Mix reduces FPR95 for ASH on UCF101 50/51 from 32.14% to 10.68%, an absolute 21.46-point improvement.
- On Kinetics-600 as ID, ReAct++ reduces FPR95 by up to 14.43 points versus the unimodal baseline, and on HMDB51 as ID, ASH++ reduces FPR95 by up to 23.38 points.
- Average L1 prediction distance between ID and OOD rises after A2D training, correlating with OOD performance gains across datasets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add A2D training and NP-Mix to ASH | ASH · UCF101 50/51 Near-OOD | FPR95: 32.14 -> 10.68 | Helped |
| Add A2D training and NP-Mix to VIM | VIM · HMDB51 Far-OOD (average) | FPR95: 16.73 -> 11.77 | Helped |
| Add A2D training and NP-Mix to GEN | GEN · Kinetics-600 Far-OOD (average) | AUROC: 80.30 -> 82.67 | Helped |
| Use L1, L2, or Wasserstein instead of Hellinger distance in A2D (ablation) | A2D with Energy · HMDB51 25/26 Near-OOD | AUROC: Baseline 87.46; L1 88.52; L2 88.84; Wasserstein 88.61; Hellinger 88.91 | Mixed |
| Replace NP-Mix with VOS (ablation) | A2D with Energy · HMDB51 25/26 Near-OOD | AUROC: 87.46 -> 87.94 | Helped |
| Replace NP-Mix with NPOS (ablation) | A2D with Energy · HMDB51 25/26 Near-OOD | AUROC: 87.46 -> 87.59 | Helped |
| Replace NP-Mix with Mixup (ablation) | A2D with Energy · HMDB51 25/26 Near-OOD | AUROC: 87.46 -> 87.77 | Helped |
| Use NP-Mix rather than VOS, NPOS, or Mixup (ablation) | A2D with Energy · HMDB51 25/26 Near-OOD | AUROC: VOS 87.94; NPOS 87.59; Mixup 87.77; NP-Mix 88.91 | Helped |
Surprising findings
- Simply adding modalities (video plus optical flow) substantially improves unimodal OOD methods even without any OOD-specific training.
- Mahalanobis-based scores degrade on several Near-OOD settings even after A2D+NP-Mix training, e.g., FPR95 rises on HMDB51 (40.31 to 44.88).
Limitations
- Near-OOD performance and performance on datasets with many classes still leave room for improvement.
- Experiments are limited to action recognition with video, optical flow, and audio modalities.
Experimental setup
Datasets: EPIC-Kitchens HAC HMDB51 UCF101 Kinetics-600 Kinetics-400 VGGSound
Benchmarks: MultiOOD Near-OOD (HMDB51 25/26, UCF101 50/51, EPIC-Kitchens 4/4, Kinetics-600 129/100) MultiOOD Far-OOD (HMDB51 and Kinetics-600 as ID)
Metrics: FPR95 AUROC ID ACC
Baselines: MSP Energy MaxLogit Mahalanobis ReAct ASH GEN KNN VIM LogitNorm VOS NPOS
Builds on: Energy NPOS VOS LogitNorm Generalized Entropy
Closest papers by meaning
- Extremely Simple Multimodal Outlier Synthesis for Out-of-Distribution Detection and Segmentation (NeurIPS 2025 · similarity 0.93)
- DPU: Dynamic Prototype Updating for Multimodal Out-of-Distribution Detection (CVPR 2025 · similarity 0.93)
- MetaOOD: Automatic Selection of OOD Detection Models (ICLR 2025 · similarity 0.90)
- Image Background Serves as Good Proxy for Out-of-distribution Data (ICLR 2024 · similarity 0.90)
- Rethinking the Evaluation of Out-of-Distribution Detection: A Sorites Paradox (NeurIPS 2024 · similarity 0.90)
- ImageNet-OOD: Deciphering Modern Out-of-Distribution Detection Algorithms (ICLR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.