Research map / Detection and segmentation / Open-set and few-shot detection
Open-vocabulary object detection
Detect novel categories by distilling vision-language knowledge into detectors.
96 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2021–2026) · an approach in Open-set and few-shot detection, keyword cluster “vocabulary · language · text”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 2 of 4 tracked benchmarks.
42 papers in 2023–24 and 46 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 7.1% to 8.3%.
Open questions the data points to
- LVIS MiniVal (AP_r): best went from 34.5 (up to 2024) to 37.3 in 2025 — LLMDet: Learning Strong Open-Vocabulary Object Detectors under the Supervision of Large Language Models (CVPR 2025 · 41 citations)
- LVIS minival (AP): best went from 43.4 (up to 2024) to 44.7 in 2025 — LLMDet: Learning Strong Open-Vocabulary Object Detectors under the Supervision of Large Language Models (CVPR 2025 · 41 citations)
No longer reported by papers here: COCO (AP50, last in 2024); Objects365 (AP50, last in 2024).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2021 · VinVL: Revisiting Visual Representations in Vision-Language Models (CVPR 2021 · 894 citations)
- 2022 · RegionCLIP: Region-Based Language-Image Pretraining (CVPR 2022 · 572 citations)
- 2023 · CORA: Adapting CLIP for Open-Vocabulary Detection With Region Prompting and Anchor Pre-Matching (CVPR 2023 · 152 citations)
- 2024 · YOLO-World: Real-Time Open-Vocabulary Object Detection (CVPR 2024 · 721 citations)
- 2025 · LLMDet: Learning Strong Open-Vocabulary Object Detectors under the Supervision of Large Language Models (CVPR 2025 · 41 citations)
- 2026 · DynamicEarth: How Far Are We from Open-Vocabulary Change Detection? (AAAI 2026 · 5 citations)
Earliest papers here
- VinVL: Revisiting Visual Representations in Vision-Language Models (CVPR 2021 · 894 citations)
- RegionCLIP: Region-Based Language-Image Pretraining (CVPR 2022 · 572 citations)
- Learning To Prompt for Open-Vocabulary Object Detection With Vision-Language Model (CVPR 2022 · 351 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
LVIS MiniVal · AP_r (higher is better)
6 papers, 2023–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 34.5 | Multi-modal Queried Object Detection in the Wild (NeurIPS 2023 · 7 citations) |
| 2025 | 37.3 | LLMDet: Learning Strong Open-Vocabulary Object Detectors under the Supervision of Large Language Models (CVPR 2025 · 41 citations) |
LVIS minival · AP (higher is better)
6 papers, 2023–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 43.4 | Multi-modal Queried Object Detection in the Wild (NeurIPS 2023 · 7 citations) |
| 2025 | 44.7 | LLMDet: Learning Strong Open-Vocabulary Object Detectors under the Supervision of Large Language Models (CVPR 2025 · 41 citations) |
COCO · AP50 (higher is better)
4 papers, 2022–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2022 | 56.6 | Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detection (NeurIPS 2022 · 69 citations) |
| 2023 | 59.1 | Learning To Detect and Segment for Open Vocabulary Object Detection (CVPR 2023 · 24 citations) |
Objects365 · AP50 (higher is better)
4 papers, 2022–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2022 | 22.3 | Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detection (NeurIPS 2022 · 69 citations) |
| 2024 | 34.4 | Domain Adaptation for Large-Vocabulary Object Detectors (NeurIPS 2024 · 0 citations) |
What has been tried
64 papers here report 799 measured changes (a component added, removed or swapped, compared with a baseline): helped 76%, no clear effect 6%, hurt 17%, mixed 1%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add GQA to O365 pre-training | LVIS minival zero-shot · AP | 23.5 -> 31.9 | Helped | YOLO-World: Real-Time Open-Vocabulary Object Detection (CVPR 2024 · 721 citations) |
| add image-level caption generation alongside grounding | LVIS minival · AP | 43.8 -> 44.0 | Helped | LLMDet: Learning Strong Open-Vocabulary Object Detectors under the Supervision of Large Language Models (CVPR 2025 · 41 citations) |
| Add GoldG training data | LVISmini · APall | 30.8 -> 41.4 | Helped | DetCLIPv3: Towards Versatile Generative Open-vocabulary Object Detection (CVPR 2024 · 39 citations) |
| add BCP only | OV-COCO · AP50 | 52.3 -> 52.5 | No clear effect | Learning Background Prompts to Discover Implicit Knowledge for Open Vocabulary Object Detection (CVPR 2024 · 29 citations) |
| Add VisGT during LAE-1M pre-training | DIOR open-set · AP50 | 83.6 -> 85.5 | Helped | Locate Anything on Earth: Advancing Open-Vocabulary Object Detection for Remote Sensing Community (AAAI 2025 · 27 citations) |
| Add self-training on novel categories | COCO open-vocabulary (novel categories) · AP50box | 37.1 -> 40.3 | Helped | InstaGen: Enhancing Object Detection by Training on Synthetic Dataset (CVPR 2024 · 27 citations) |
| Add Text MLR | OV-LVIS · AP_r | 16.4 -> 17.6; +1.2 | Helped | Simple Image-Level Classification Improves Open-Vocabulary Object Detection (AAAI 2024 · 25 citations) |
| Add RAL to OADP without RAF | COCO · APN50 | 30.0 -> 31.5 | Helped | Retrieval-Augmented Open-Vocabulary Object Detection (CVPR 2024 · 22 citations) |
| Use an external RPN instead of training the backbone for region proposals | COCO-OVD val · APnovel50 | 25.4 -> 31.4 | Helped | Taming Self-Training for Open-Vocabulary Object Detection (CVPR 2024 · 18 citations) |
| Increase the number of hard-negative captions in the vocabulary | Difficulty-based and attribute-based FG-OVD benchmarks · mAP | Hurt | The Devil is in the Fine-Grained Details: Evaluating Open-Vocabulary Object Detectors for Fine-Grained Understanding (CVPR 2024 · 16 citations) | |
| Add multi-level decoding | SWIG-HOI Full · mAP | 11.45 -> 12.87 | Helped | Exploring the Potential of Large Foundation Models for Open-Vocabulary HOI Detection (CVPR 2024 · 15 citations) |
| add the scene-adaptive prompt generator | OV-COCO · mAP50 (novel) | 27.6 -> 32.8 | Helped | Scene-adaptive and Region-aware Multi-modal Prompt for Open Vocabulary Object Detection (CVPR 2024 · 15 citations) |
What reviewers will expect
Datasets and baselines named in the 19 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- LVIS · 7 of 19 papers
- COCO · 6 of 19 papers
- V3Det · 3 of 19 papers
- GoldG · 3 of 19 papers
- Visual Genome · 2 of 19 papers
- ODinW13 · 2 of 19 papers
- RefCOCO · 2 of 19 papers
- DIOR · 2 of 19 papers
Baselines
- Grounding DINO · 6 of 19 papers
- CORA · 5 of 19 papers
- Detic · 4 of 19 papers
- RegionCLIP · 4 of 19 papers
- CoDet · 3 of 19 papers
- Faster R-CNN · 3 of 19 papers
- GLIP · 3 of 19 papers
- DITO · 2 of 19 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 88 papers; from OpenAlex and the papers' affiliations).
- University of Chinese Academy of Sciences · 5 papers
- The University of Hong Kong · 5 papers
- Shanghai Jiao Tong University · 5 papers
- Google · 5 papers
- Beihang University · 4 papers
- Nanyang Technological University · 4 papers
- SenseTime · 4 papers
- ByteDance · 4 papers
- Chinese Academy of Sciences · 3 papers
- Shanghai AI Laboratory · 3 papers
Use it
Download BibTeX of 37 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Open-vocabulary semantic segmentation (93 papers)
- Open-vocabulary 3D and world perception (41 papers)
Elsewhere in the topic
- Object-level representation learning (219 papers)
- Interactive and referring segmentation (161 papers)
- Open-set and unknown object detection (160 papers)
- Amodal and open-world instance segmentation (137 papers)
- Industrial anomaly detection (117 papers)
- Unsupervised object and part discovery (114 papers)
- Domain-adaptive object detection (103 papers)
- Object-centric learning theory and video (86 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).