Research map / Label-efficient and robust learning / OOD and anomaly detection
DPU: Dynamic Prototype Updating for Multimodal Out-of-Distribution Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DPU, a plug-and-play multimodal OOD detection framework that dynamically updates class prototypes and sample-specific prediction-discrepancy intensification to handle intra-class variability in ID data.
Problem
Multimodal out-of-distribution detection assumes all samples of a class are cohesive, so uniformly amplifying cross-modal prediction discrepancy hurts ID accuracy and detection when intra-class variability exists.
Approach
Dynamically update per-class prototypes with batch-variance-dependent update rates, then scale each sample's multimodal prediction discrepancy by its similarity to its class prototype (pro-ratio discrepancy intensification), combined with contrastive training (CSCT) and adaptive outlier synthesis; it is applied on top of nine base OOD methods.
Key results
- On HMDB51 with Energy, DPU reduces FPR95 from 43.36 to 35.07 and raises AUROC from 87.46 to 89.52 and ID ACC from 89.32 to 92.16.
- DPU gives around 10% gains across metrics for Near-OOD and up to 80% in certain metrics for Far-OOD detection.
- Adaptive intensification outperforms all fixed intensification ratios (0.1, 0.3, 0.5, 0.7) in the Mahalanobis ablation on UCF101 and Kinetics-600.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply DPU to Energy | Energy OOD detector · HMDB51 25/26 near-OOD | AUROC: 87.46 -> 89.52 | Helped |
| Apply DPU to Energy | Energy OOD detector · UCF101 50/51 near-OOD | FPR95: 22.52 -> 7.48 | Mixed |
| Apply DPU to Energy | Energy OOD detector · HMDB51 ID; Kinetics600 OOD | AUROC: 92.48 -> 95.81 | Helped |
| Replace adaptive discrepancy intensification with a fixed ratio (ablation) | Mahalanobis multimodal detector · UCF101 and Kinetics-600 near-OOD | AUROC: UCF101: 97.88 (0.1), 98.44 (0.3), 97.99 (0.5), 98.28 (0.7), DPU: 98.68; Kinetics-600: 50.05 (0.1), 50.58 (0.3), 50.10 (0 | Mixed |
| Use fixed discrepancy intensification ratio 0.1 instead of adaptive intensity (ablation) | Mahalanobis multimodal detector · UCF101 near-OOD | FPR95: 10.19 vs 6.60 | Hurt |
| Use fixed discrepancy intensification ratio 0.3 instead of adaptive intensity (ablation) | Mahalanobis multimodal detector · UCF101 and Kinetics-600 near-OOD | AUROC: UCF101: 98.44; Kinetics-600: 50.58; DPU: 98.68 and 54.17 | Mixed |
| Use fixed discrepancy intensification ratio 0.5 instead of adaptive intensity (ablation) | Mahalanobis multimodal detector · UCF101 and Kinetics-600 near-OOD | AUROC: UCF101: 97.99; Kinetics-600: 50.10; DPU: 98.68 and 54.17 | Mixed |
| Use fixed discrepancy intensification ratio 0.7 instead of adaptive intensity (ablation) | Mahalanobis multimodal detector · UCF101 and Kinetics-600 near-OOD | AUROC: UCF101: 98.28; Kinetics-600: 50.81; DPU: 98.68 and 54.17 | Mixed |
| Use CSCT contrastive training (ablation) | MSP multimodal detector · UCF101 and Kinetics-600 near-OOD | FPR95 | Helped |
| Use CSCT contrastive training (ablation) | MSP multimodal detector · UCF101 and Kinetics-600 near-OOD | ID ACC | Helped |
Surprising findings
- Uniformly intensifying prediction discrepancy across all ID samples can degrade ID accuracy, and intensifying it on class-center samples causes substantial confusion.
Limitations
- Effectiveness may vary on significantly larger or more complex multimodal datasets, raising scalability concerns.
- Not yet optimized for real-time applications.
- Cross-domain OOD detection is not explored.
Experimental setup
Datasets: HMDB51 UCF101 Kinetics-600 HAC EPIC-Kitchens
Benchmarks: HMDB51 UCF101 Kinetics-600 HAC EPIC-Kitchens
Metrics: FPR95 AUROC ID ACC
Baselines: MSP Energy Maxlogit Mahalanobis ReAct ASH GEN KNN VIM Agree-to-Disagree (A2D) AN (A2D + NP-Mix) InfoNCE
Builds on: Agree-to-Disagree (A2D) NP-Mix invariant risk minimization InfoNCE Energy Mahalanobis
Closest papers by meaning
- MultiOOD: Scaling Out-of-Distribution Detection for Multiple Modalities (NeurIPS 2024 · similarity 0.93)
- Extremely Simple Multimodal Outlier Synthesis for Out-of-Distribution Detection and Segmentation (NeurIPS 2025 · similarity 0.90)
- Learning with Mixture of Prototypes for Out-of-Distribution Detection (ICLR 2024 · similarity 0.89)
- Enhanced OoD Detection through Cross-Modal Alignment of Multi-Modal Representations (CVPR 2025 · similarity 0.88)
- The Best of Both Worlds: On the Dilemma of Out-of-distribution Detection (NeurIPS 2024 · similarity 0.88)
- Spurious-Aware Prototype Refinement for Reliable Out-of-Distribution Detection (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.