Research map / Detection and segmentation / Open-set and few-shot detection
UMB: Understanding Model Behavior for Open-World Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UMB, a method for open-world object detection that models text-attribute probabilities on OWL-ViT to detect unknown objects and explain which known classes and attributes drive the unknown predictions.
Problem
Open-world object detection methods recall unknown objects but do not explain why the model predicts them as unknown; the paper aims to understand the model's behavior when predicting the unknown class.
Approach
Model the relation between text-attribute similarity and positive-sample probability (empirical probability via Gaussian/Weibull mixture fitting), combine it with in-distribution and out-of-distribution probabilities to score unknowns, and then infer the most similar known class and the most influential attribute for each unknown prediction.
Key results
- UMB-Gm-L reaches 20.5 unknown mAP on RWD overall, a 5.3 mAP gain over the previous SOTA.
- On Surgery, UMB-Wb-L achieves 25.6 unknown mAP versus 12.0 for FOMO-L.
- Similarity inference between unknown objects and known classes is reported as about 90% accurate on overlapping predictions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use UMB-Gm-L instead of FOMO-L | OWL-ViT-L/14 · RWD Overall Task1 unknown categories | mAP: 15.2 -> 20.5 | Helped |
| use UMB-Gm-L instead of FOMO-L | OWL-ViT-L/14 · RWD Overall Task2 unknown categories | mAP: 27.4 -> 32.7 | Helped |
| add in-distribution probability to mean embedding and OOD probability (ablation) | UMB unknown inference · RWD Overall | UAP (mAP): 9.7 -> 15.5 | Helped |
| add in-distribution probability to mean embedding and OOD probability (ablation) | UMB unknown inference · RWD Aerial | UAP (mAP): 4.8 -> 3.8 | Hurt |
| add empirical probability to in-distribution and OOD probabilities (ablation) | UMB unknown inference · RWD Overall | UAP (mAP): 15.5 -> 20.5 | Helped |
| use Gaussian rather than Weibull distribution fitting (ablation) | UMB · RWD Overall | UAP (mAP): 19.3 -> 20.5 | Helped |
| use Weibull rather than Gaussian distribution fitting (ablation) | UMB · RWD Surgery | UAP (mAP): 24.5 -> 25.6 | Helped |
| add linear interpolation to the original distribution (ablation) | UMB probability mixture modeling · RWD Overall | mAP: 18.3 -> 18.4 | No clear effect |
| use sliding-window filtering on the original distribution (ablation) | UMB probability mixture modeling · RWD Overall | mAP: 18.3 -> 18.4 | No clear effect |
| vary balance parameter α (ablation) | UMB unknown inference · RWD unknown-category recall | recall | No clear effect |
Surprising findings
- Unknown recall alone can reward greedy background predictions; the paper shows adding ID probability raised recall by 44.1 on Aerial while unknown mAP dropped by 1.
Limitations
- Performance ceiling is bounded by the underlying OWL-ViT open-vocabulary detector since weights are not trained.
- Sensitivity to the balance parameter alpha on datasets far from OWL-ViT pretraining (e.g., Aerial, Medical, Surgery).
- Requires extensive data for visual-text alignment; fine-tuning on application data adds annotation cost.
Experimental setup
Datasets: RWD (Aquatic, Aerial, Game, Medical, Surgery)
Benchmarks: RWD
Metrics: mAP unknown mAP (UAP) unknown recall (URE)
Baselines: FOMO Base+GT Base-FS OW-DETR ORE
Builds on: OWL-ViT FOMO ORE OW-DETR
Closest papers by meaning
- OW-OVD: Unified Open World and Open Vocabulary Object Detection (CVPR 2025 · similarity 0.92)
- Open-World Objectness Modeling Unifies Novel Object Detection (CVPR 2025 · similarity 0.90)
- Detecting Open World Objects via Partial Attribute Assignment (CVPR 2025 · similarity 0.90)
- Semi-supervised Open-World Object Detection (AAAI 2024 · similarity 0.89)
- OW-VAP: Visual Attribute Parsing for Open World Object Detection (ICML 2025 · similarity 0.89)
- Hyp-OW: Exploiting Hierarchical Structure Learning with Hyperbolic Distance Enhances Open World Object Detection (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.