Research map / Detection and segmentation / Open-set and few-shot detection
Learning Background Prompts to Discover Implicit Knowledge for Open Vocabulary Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LBP, which learns multiple background category-specific prompts discovered from background proposals (plus online background object discovery and inference probability rectification) to improve open-vocabulary object detection on OV-COCO and OV-LVIS.
Problem
Knowledge-distillation-based open vocabulary object detectors represent all background proposals with a single background embedding, losing implicit background knowledge about unseen categories and overfitting to base classes.
Approach
Estimate the number of latent background categories via k-means over CLIP features of background proposals, learn a learnable context prompt per underlying category, discover pseudo-labelled background objects online, and rectify inference probabilities for novel classes to account for overlap with these background categories.
Key results
- On OV-COCO under the V-OVD setting, LBP reaches 54.3 AP50 and 35.9 novel AP50 versus BARON's 53.5 and 34.0.
- On OV-LVIS, LBP‡ achieves 24.1 APr for object detection versus 23.2 for BARON‡, and 29.9 AP overall versus 29.5.
- Ablation on OV-COCO shows the full model with BCP, BOD and IPR reaches 37.8 APn50, improving over the single-embedding baseline's 35.8.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add BCP only (ablation) | Faster R-CNN OVD baseline · OV-COCO | AP50: 52.3 -> 52.5 | No clear effect |
| add BOD only (ablation) | Faster R-CNN OVD baseline · OV-COCO | AP50: 52.3 -> 52.4 | No clear effect |
| add BCP and IPR (ablation) | Faster R-CNN OVD baseline · OV-COCO | APn50: 35.8 -> 36.8 | Helped |
| add BOD to BCP + IPR (ablation) | Faster R-CNN OVD baseline · OV-COCO | APn50: 36.8 -> 37.8 | Helped |
| use Lbcp only instead of final background loss L′bcp (ablation) | LBP background interpretation · OV-COCO | AP50: 53.2 -> 53.1 | Hurt |
| use Lrlx only instead of final background loss L′bcp (ablation) | LBP background interpretation · OV-COCO | AP50: 53.2 -> 52.8 | Hurt |
| use DetPro soft background loss instead of LBP background interpretation (ablation) | LBP · OV-COCO | AP50: 53.2 -> 52.6 | Hurt |
| replace multiple background category prompts with a single background embedding (ablation) | LBP background interpretation · OV-COCO | AP50: 53.2 -> 52.3 | Hurt |
| add LBP background prompting and discovery framework | BARON · OV-COCO | APn50: 34.0 -> 35.9 | Helped |
| add LBP background prompting and discovery framework | BARON · OV-COCO | APb50: 60.4 -> 60.8 | Helped |
| add LBP | BARON‡ · OV-LVIS object detection | APr: 23.2 -> 24.1 | Helped |
| add LBP | BARON‡ · OV-LVIS instance segmentation | APr: 22.6 -> 23.7 | Helped |
Surprising findings
- Using only the background category prompt module degrades novel-class performance, attributed to conceptual overlap between estimated background categories and novel classes.
Limitations
- Relies on the number of background categories being estimated via k-means and a pre-defined hyper-parameter na.
- Gains on OV-LVIS overall AP are modest relative to the baselines (e.g., 29.1 vs 29.5 for BARON without prompts).
Experimental setup
Datasets: MS-COCO LVIS
Benchmarks: OV-COCO OV-LVIS
Metrics: AP50 APn50 APb50 APr APc APf AP
Baselines: Detic Object-centric-OVD OV-DETR RegionCLIP ViLD OADP BARON DetPro
Builds on: ViLD DetPro BARON VL-PLM CoOp
Closest papers by meaning
- CAKE: Category Aware Knowledge Extraction for Open-Vocabulary Object Detection (AAAI 2025 · similarity 0.90)
- How to Evaluate the Generalization of Detection? A Benchmark for Comprehensive Open-Vocabulary Detection (AAAI 2024 · similarity 0.90)
- Cyclic Contrastive Knowledge Transfer for Open-Vocabulary Object Detection (ICLR 2025 · similarity 0.90)
- Taming Self-Training for Open-Vocabulary Object Detection (CVPR 2024 · similarity 0.89)
- DitHub: A Modular Framework for Incremental Open-Vocabulary Object Detection (NeurIPS 2025 · similarity 0.89)
- Open-Det: An Efficient Learning Framework for Open-Ended Detection (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.