Research map / Detection and segmentation / Open-set and few-shot detection
UN-DETR: Promoting Objectness Learning via Joint Supervision for Unknown Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UN-DETR, a transformer-based unknown object detector that learns a jointly supervised Instance Presence Score from positional and categorical latent spaces, with one-to-many assignment, unbiased query selection, and IPS-guided post-processing.
Problem
Unknown Object Detection (UOD) requires detecting objects of unseen categories; prior methods learn objectness from only localization or only classification signals, yielding low recall and precision for unknown objects.
Approach
Learn an Instance Presence Score (IPS) via a dedicated predictor supervised jointly by GIoU-based positional objectness and summed known-class logits, add one-to-many (one-to-two) assignment for IPP training, replace the two-stage query-selection classification head with the IPP (Unbiased Query Selection), and use IPS for NMS and known/unknown separation, with unsupervised pretraining using Select
Key results
- On COCO-OOD, UN-DETR reaches U-AP 47.0, U-F1 54.9 and U-PRE 54.5, the best on those metrics, with mAP 47.2 on VOC-test.
- On COCO-Mixed, UN-DETR achieves U-AP 18.9 versus 14.8 for the second-best method (UnSniffer*), but U-REC 32.8 trails ORE's 59.2.
- Ablation on COCO-OOD: removing unsupervised pretraining drops U-PRE from 54.5 to 35.0.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| jointly supervise IPP with classification and regression information rather than classification alone (ablation) | UN-DETR · COCO-OOD | U-AP: 17.2 -> 47.0 | Helped |
| jointly supervise IPP with classification and regression information rather than regression alone (ablation) | UN-DETR · COCO-OOD | U-AP: 40.9 -> 47.0 | Helped |
| add one additional set of positive queries for IPP training (ablation) | UN-DETR · COCO-OOD | U-AP: 46.0 -> 47.0 | Helped |
| use one-to-three rather than one-to-two assignment (ablation) | UN-DETR · COCO-OOD | U-F1: 54.9 -> 53.7 | Hurt |
| use IPS rather than the original classification head for query filtering (ablation) | UN-DETR · COCO-OOD | U-AP: 39.7 -> 47.0 | Helped |
| use jointly supervised IPS rather than regression-only IPP for query filtering (ablation) | UN-DETR · COCO-OOD | U-AP: 45.9 -> 47.0 | Helped |
| use IPS-guided NMS rather than original NMS (ablation) | UN-DETR · COCO-OOD | U-AP: 46.7 -> 47.0 | Helped |
| add unsupervised pretraining (ablation) | UN-DETR · COCO-OOD | U-PRE: 35.0 -> 54.5 | Helped |
| add unsupervised pretraining (ablation) | UN-DETR · COCO-OOD | U-AP: 45.8 -> 47.0 | Helped |
| use UN-DETR rather than UnSniffer | unknown-object detection · COCO-OOD | U-F1: 47.9 -> 54.9 | Helped |
| use UN-DETR rather than ORE | unknown-object detection · COCO-OOD | U-REC: 78.2 -> 55.3 | Hurt |
| use UN-DETR rather than UnSniffer* | unknown-object detection · COCO-Mixed | U-F1: 26.7 -> 24.7 | Hurt |
Surprising findings
- Grounding DINO, evaluated zero-shot with an "unknown object" prompt, performs much worse on COCO-OOD (U-AP 26.0) than UN-DETR, and only improves when known category names are added to the prompt.
Limitations
- The training and evaluation data are small (VOC, 16K images, 20 categories), which limits the method according to the authors.
- Inference speed is not optimized for real-world deployment.
Experimental setup
Datasets: PASCAL VOC
Benchmarks: COCO-OOD COCO-Mixed PASCAL VOC
Metrics: mAP U-AP U-F1 U-PRE U-REC
Baselines: MSP Mahalanobis Energy score VOS ORE OW-DETR PROB UnSniffer Grounding DINO
Builds on: Deformable DETR DETRreg PROB UnSniffer two-stage Deformable DETR
Closest papers by meaning
- OW-DETR: Open-World Detection Transformer (CVPR 2022 · similarity 0.89)
- Learning Latent Concepts for Detecting Out-of-Distribution Objects (CVPR 2026 · similarity 0.88)
- Spatially Invariant Unsupervised Object Detection with Convolutional Neural Networks (AAAI 2019 · similarity 0.87)
- DETR With Additional Global Aggregation for Cross-Domain Weakly Supervised Object Detection (CVPR 2023 · similarity 0.87)
- MS-DETR: Efficient DETR Training with Mixed Supervision (CVPR 2024 · similarity 0.87)
- DetKDS: Knowledge Distillation Search for Object Detectors (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.