Research map / Detection and segmentation / Open-set and few-shot detection
Extremely Simple Multimodal Outlier Synthesis for Out-of-Distribution Detection and Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Feature Mixing, a training-time multimodal outlier synthesis method that swaps random feature dimensions between modalities and maximizes outlier entropy, plus a new CARLA-OOD segmentation dataset.
Problem
Multimodal out-of-distribution detection and segmentation suffer from overconfidence on OOD inputs because no outlier supervision is available, and existing outlier synthesis methods are either unimodal, slow, or introduce noisy samples.
Approach
Generate multimodal outliers in feature space by randomly swapping N feature dimensions between two modality embeddings, then maximize prediction entropy on these outliers during training; the method is modality-agnostic and far cheaper than nearest-neighbor synthesis (NP-Mix) or VOS.
Key results
- On SemanticKITTI, A2D + Feature Mixing reaches FPR@95 31.76, AUROC 92.83 and AUPR 61.99 versus 49.02 / 91.12 / 55.44 for A2D alone.
- On CARLA-OOD, Feature Mixing improves Late Fusion FPR@95 by 72.98% and AUROC by 35.74%.
- Feature Mixing gives a 10x speedup over NP-Mix for OOD detection (0.058 s vs 0.545 s) and 370x for segmentation (0.013 s vs 4.81 s).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Feature Mixing to Late Fusion | Multimodal OOD segmentation · SemanticKITTI | FPR@95: 53.43 -> 38.10 | Helped |
| Add Feature Mixing to Late Fusion | Multimodal OOD segmentation · nuScenes | AUROC: 82.60 -> 86.83 | Helped |
| Add Feature Mixing to Late Fusion | Multimodal OOD segmentation · CARLA-OOD | FPR@95: 98.83 -> 25.85 | Helped |
| Add Feature Mixing to A2D | A2D segmentation framework · SemanticKITTI | FPR@95: 49.02 -> 31.76 | Helped |
| Use Feature Mixing instead of NP-Mix for multimodal OOD synthesis | MultiOOD with HMDB51 as ID · MultiOOD average | AUROC: 93.89 -> 94.17 | Helped |
| Add Feature Mixing to the tri-modal video, optical-flow, and audio setup (ablation) | MultiOOD · EPIC-Kitchens (tri-modal) | AUROC: 74.95 -> 77.23 | Helped |
| Use video-only Feature Mixing, swapping dimensions between embedding halves (ablation) | MultiOOD · HMDB51 (unimodal) | AUROC: 83.62 -> 84.41 | Helped |
| Replace MaxLogit with MSP as the OOD score (ablation) | A2D + Feature Mixing · SemanticKITTI | FPR@95: 31.76 -> 32.93 | Hurt |
| Replace MaxLogit with Energy as the OOD score (ablation) | A2D + Feature Mixing · SemanticKITTI | AUROC: 92.83 -> 92.87 | No clear effect |
| Replace MaxLogit with Entropy as the OOD score (ablation) | A2D + Feature Mixing · SemanticKITTI | FPR@95: 31.76 -> 33.00 | Hurt |
| Replace MaxLogit with GEN as the OOD score (ablation) | A2D + Feature Mixing · SemanticKITTI | AUROC: 92.83 -> 93.00 | Helped |
| Designate ground classes as OOD and use Feature Mixing with A2D (ablation) | A2D segmentation framework · SemanticKITTI (ground classes OOD) | FPR@95: 71.15 -> 36.30 | Helped |
Surprising findings
- Mixup is very fast but performs poorly on OOD segmentation and detection, and on CARLA-OOD it increases FPR@95 relative to Late Fusion (99.23 vs 98.83).
Limitations
- Random selection of feature dimensions may not target the most informative regions for outlier synthesis.
- CARLA-OOD is synthetic, so transfer to real-world OOD objects is not established by the paper.
- Multimodal OOD segmentation treats vehicle classes as OOD on real datasets, which may leak OOD class appearance during training.
Experimental setup
Datasets: SemanticKITTI nuScenes KITTI-CARLA MultiOOD HMDB51 UCF101 Kinetics-600 HAC EPIC-Kitchens
Benchmarks: SemanticKITTI nuScenes CARLA-OOD MultiOOD
Metrics: FPR@95 AUROC AUPR mIoU_c ACC computational time
Baselines: Late Fusion xMUDA A2D Mixup NP-Mix VOS NPOS
Builds on: Mixup VOS NP-Mix Agree-to-Disagree (A2D) PMF
Closest papers by meaning
- MultiOOD: Scaling Out-of-Distribution Detection for Multiple Modalities (NeurIPS 2024 · similarity 0.93)
- DPU: Dynamic Prototype Updating for Multimodal Out-of-Distribution Detection (CVPR 2025 · similarity 0.90)
- Image Background Serves as Good Proxy for Out-of-distribution Data (ICLR 2024 · similarity 0.89)
- GOOD: Training-Free Guided Diffusion Sampling for Out-of-Distribution Detection (NeurIPS 2025 · similarity 0.88)
- Diffusion-based Semantic Outlier Generation via Nuisance Awareness for Out-of-Distribution Detection (AAAI 2025 · similarity 0.88)
- Envisioning Outlier Exposure by Large Language Models for Out-of-Distribution Detection (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.