Research map / Detection and segmentation / Open-set and few-shot detection
RUNA: Object-Level Out-of-Distribution Detection via Regional Uncertainty Alignment of Multimodal Representations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RUNA, a CLIP-based post-hoc object-level OOD detector that fuses global and regional image embeddings and scores objects by max text similarity, with few-shot fine-tuning of the projection layer.
Problem
Object-level out-of-distribution detection for object detectors, where detectors give overconfident predictions on unknown objects and image-level CLIP OOD methods lose local context when applied to regions.
Approach
A dual image-encoder (global image plus cropped, blurred region) CLIP design whose fused embedding is scored by maximum similarity to ID class prompts, with a 10-shot contrastive fine-tune of only the regional projection layer, applied post hoc to a frozen Faster R-CNN.
Key results
- On BDD-100K as ID with OpenImages as OOD, RUNA reaches FPR95 9.95% and AUROC 96.76%, versus 16.04% FPR95 for the prior best SAFE.
- On BDD-100K with MSCOCO as OOD, RUNA achieves FPR95 16.85% (SAFE 32.56%).
- On PASCAL-VOC with MSCOCO as OOD, RUNA achieves FPR95 30.67% and AUROC 92.48%, beating TIB at 41.55% FPR95; on VOC/OpenImages SAFE has lower FPR95 (20.36% vs RUNA 26.07%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RUNA instead of SAFE | BDD-100K ID, OpenImages OOD · BDD-100K ID, OpenImages OOD | FPR95: 16.04 -> 9.95 | Helped |
| Use RUNA instead of SAFE | BDD-100K ID, MSCOCO OOD · BDD-100K ID, MSCOCO OOD | FPR95: 32.56 -> 16.85 | Helped |
| Use RUNA instead of SAFE | PASCAL-VOC ID, OpenImages OOD · PASCAL-VOC ID, OpenImages OOD | FPR95: 20.36 -> 26.07 | Hurt |
| Use RUNA instead of SAFE | PASCAL-VOC ID, MSCOCO OOD · PASCAL-VOC ID, MSCOCO OOD | FPR95: 47.40 -> 30.67 | Helped |
| Add the global encoder to the regional encoder (ablation) | RUNA zero-shot regional encoder · BDD-100K ID, OpenImages/MSCOCO OOD | FPR95: 58.09/62.17 -> 29.42/36.96 | Helped |
| Add the global encoder to the regional encoder (ablation) | RUNA zero-shot regional encoder · PASCAL-VOC ID, OpenImages/MSCOCO OOD | FPR95: 42.75/55.26 -> 38.97/51.37 | Helped |
| Add few-shot fine-tuning (ablation) | RUNA regional encoder · BDD-100K ID, OpenImages/MSCOCO OOD | FPR95: 58.09/62.17 -> 15.71/22.14 | Helped |
| Add few-shot fine-tuning (ablation) | RUNA regional encoder · PASCAL-VOC ID, OpenImages/MSCOCO OOD | FPR95: 42.75/55.26 -> 30.76/34.31 | Helped |
| Increase the number of fine-tuning shots from zero to 1, then 5 and 10 (ablation) | RUNA · Few-shot fine-tuning evaluation | accuracy and robustness | Helped |
| Fine-tune on the entire dataset instead of few-shot samples (ablation) | RUNA · Few-shot fine-tuning evaluation | detection performance | No clear effect |
| Replace CLIP ViT-B/16 with larger ViT-L/14 backbones (ablation) | Zero-shot regional encoder · BDD-100K and PASCAL-VOC; OpenImages/MSCOCO OOD | FPR95 and AUROC | Mixed |
Surprising findings
- On PASCAL-VOC with OpenImages as OOD, the prior SAFE method beats RUNA on FPR95 (20.36% vs 26.07%).
- Direct summation of similarities fails to separate ID and OOD objects on VOC because CLIP cosine similarities have limited variance.
Limitations
- Detector-agnostic claim is weaker on VOC/OpenImages where SAFE is better on FPR95.
- Relies on a frozen detector and CLIP with inference cost growing with the number of regions (runtime 410 ms for 10 images with ViT-B/16).
Experimental setup
Datasets: PASCAL-VOC BDD-100K MS-COCO OpenImages
Benchmarks: OpenImages MSCOCO
Metrics: FPR95 AUROC
Baselines: MSP ODIN Mahalanobis Energy score Gram matrices Generalized ODIN CSI VOS SIREN MCM SAFE TIB
Builds on: CLIP Maximum Concept Matching (MCM) Faster R-CNN SAFE
Closest papers by meaning
- COUNTS: Benchmarking Object Detectors and Multimodal Large Language Models under Distribution Shifts (CVPR 2025 · similarity 0.90)
- DPU: Dynamic Prototype Updating for Multimodal Out-of-Distribution Detection (CVPR 2025 · similarity 0.88)
- Overcoming Shortcut Problem in VLM for Robust Out-of-Distribution Detection (CVPR 2025 · similarity 0.88)
- MultiOOD: Scaling Out-of-Distribution Detection for Multiple Modalities (NeurIPS 2024 · similarity 0.88)
- A Systematic Analysis of Out-of-Distribution Detection Under Representation and Training Paradigm Shifts (NeurIPS 2026 · similarity 0.88)
- Image Background Serves as Good Proxy for Out-of-distribution Data (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.