Research map / Detection and segmentation / Open-set and few-shot detection
OV-DQUO: Open-Vocabulary DETR with Denoising Text Query Training and Open-World Unknown Objects Supervision
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OV-DQUO, an open-vocabulary DETR that counters base-novel confidence bias using open-world unknown-object pseudo-labels with wildcard text matching, denoising text-query contrastive training, and RoQIs proposal selection.
Problem
Open-vocabulary detectors trained on base categories assign higher confidence to base classes and confuse novel objects with background, limiting novel-category detection.
Approach
Use an open-world detector (OLN) with Weibull-based foreground estimation to find unknown objects, match them to a generic wildcard text embedding ("a photo of a object"), and synthesize foreground/background noisy query-box pairs for denoising contrastive training, plus a RoQIs selection that mixes region-text similarity with objectness.
Key results
- OV-DQUO with CLIP RN50x4 reaches 45.6 AP50 on novel classes of OV-COCO, above CORA's 41.7 and CLIPSelf's 44.3.
- OV-DQUO with CLIP ViT-L/14 reaches 39.3 mAP_r on OV-LVIS, +4.4 over CLIPSelf's 34.9.
- Ablation on OV-COCO (RN50x4): open-world supervision raises novel AP50 from 41.7 to 43.3, denoising training adds 1.7 more, and RoQIs adds 0.6 for a total of 45.6.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add open-world unknown-object supervision (ablation) | OV-DQUO · OV-COCO novel | AP50: 41.7 -> 43.3 | Helped |
| Add denoising text query training (ablation) | OV-DQUO with open-world supervision · OV-COCO novel | AP50: 43.3 -> 45.0 | Helped |
| Add RoQIs selection (ablation) | OV-DQUO with open-world supervision and denoising · OV-COCO novel | AP50: 45.0 -> 45.6 | Helped |
| Use open-world supervision, denoising text query training, and RoQIs selection instead of the vanilla detector (ablation) | OV-DQUO · OV-COCO novel | AP50: 41.7 -> 45.6 | Helped |
| Use 2 rather than 1 pseudo-labeling iteration (ablation) | OV-DQUO · OV-COCO novel | AP50: 44.0 -> 45.0 | Helped |
| Increase pseudo-labeling iterations from 2 to 3 (ablation) | OV-DQUO · OV-COCO novel | AP50: 45.0 -> 44.8 | Hurt |
| Set foreground-score scaling gamma to 0, removing foreground estimation (ablation) | OV-DQUO · OV-COCO novel | AP50: 45.0 -> 43.0 | Hurt |
| Use foreground-score scaling gamma 0.5 instead of 0.0, 1.0, or 2.0 (ablation) | OV-DQUO · OV-COCO novel | AP50: 43.0; 45.0; 44.4; 44.1 | Helped |
| Vary the denoising loss weight beta from 2.0 (ablation) | OV-DQUO · OV-COCO novel | AP50: 44.8; 45.0; 44.4; 44.4 | No clear effect |
| Use simple, general wildcard text such as Thing or Object rather than intricate wildcard text (ablation) | OV-DQUO · OV-COCO novel | AP50: 44.1; 44.4 -> 44.9; 45.0 | Helped |
| Add OWD proposals to F-VLM | F-VLM · OV-COCO novel | AP50: 27.1 -> 29.3(+2.2) | Helped |
| Use OV-DQUO instead of CLIPself | OV-DQUO and CLIPself with ViT-L/14 · OV-LVIS rare | mAP_r: 34.9 -> 39.3 | Helped |
Surprising findings
- Replacing the detector's objectness-based proposal selection with region-text similarity was not the bottleneck; the authors argue a frozen VLM is an effective region classifier and confidence bias is the main issue.
- Simple generic wildcard words such as 'object' and 'thing' outperformed more specific wildcards such as 'Foreground Region' and 'Salient Object'.
Limitations
- Open-world detector (OLN) misses some novel objects, and produces non-object regions such as trees and buildings, which adds pseudo-label noise.
- Iterative pseudo-labeling more than 2 iterations degrades novel-class performance on OV-COCO.
- Integration of open-vocabulary detection with open-world detection into one end-to-end framework is left for future work.
Experimental setup
Datasets: COCO LVIS Objects365
Benchmarks: OV-COCO OV-LVIS COCO Objects365
Metrics: AP50 novel AP50 all mAP rare AP AP75
Baselines: CORA CLIPSelf SAS-Det CoDet BARON-KD F-VLM RegionCLIP CLIM
Builds on: DINO CORA OLN PROB CLIP
Closest papers by meaning
- Cyclic Contrastive Knowledge Transfer for Open-Vocabulary Object Detection (ICLR 2025 · similarity 0.87)
- CAKE: Category Aware Knowledge Extraction for Open-Vocabulary Object Detection (AAAI 2025 · similarity 0.87)
- Language-conditioned Detection Transformer (CVPR 2024 · similarity 0.86)
- ProxyDet: Synthesizing Proxy Novel Classes via Classwise Mixup for Open-Vocabulary Object Detection (AAAI 2024 · similarity 0.86)
- SHiNe: Semantic Hierarchy Nexus for Open-vocabulary Object Detection (CVPR 2024 · similarity 0.86)
- OVMR: Open-Vocabulary Recognition with Multi-Modal References (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.