Research map / Detection and segmentation / Open-set and few-shot detection
Simple Open-Vocabulary Object Detection with Vision Transformers
Matthias Minderer, Alexey Gritsenko, Austin Stone, Maxim Neumann, Dirk Weissenborn, Alexey Dosovitskiy, Aravindh Mahendran, Anurag Arnab, Mostafa Dehghani, Zhuoran Shen, Xiao Wang, Xiaohua Zhai and 2 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Visual Genome to Open Images training data (ablation) | OWL-ViT R26+B/32 (LVIS) · LVIS rare | APr: -5.7 | Improved |
| add Open Images to Visual Genome training data (ablation) | OWL-ViT R26+B/32 (LVIS) · LVIS rare | APr: -14.0 | Improved |
| use lower text encoder learning rate than image encoder (ablation) | OWL-ViT R26+B/32 (LVIS) · LVIS rare | APr: -8.5 | Improved |
| ensemble predictions over prompt templates at inference (ablation) | OWL-ViT R26+B/32 (LVIS) · LVIS rare | APr: -5.5 | Improved |
| add random prompts in training and inference (ablation) | OWL-ViT R26+B/32 (LVIS) · LVIS rare | APr: -1.3 | Improved |
| add random pseudo-negative labels (ablation) | OWL-ViT R26+B/32 (LVIS) · LVIS rare | APr: -2.8 | Improved |
| add mosaic image augmentation (ablation) | OWL-ViT R26+B/32 (LVIS) · LVIS rare | APr: -1.5 | Improved |
| train 3x longer without mosaics (ablation) | OWL-ViT R26+B/32 (LVIS) · LVIS rare | APr: -1.5 -> -3.6 | Hurt |
| add location bias to box predictor (ablation) | OWL-ViT R26+B/32 (LVIS) · LVIS rare | APr: -1.1 | Improved |
| merge overlapping instances (ablation) | OWL-ViT R26+B/32 (LVIS) · LVIS rare | APr: -1.3 | Improved |
| remove OI crowd instances (ablation) | OWL-ViT R26+B/32 (LVIS) · LVIS rare | APr: 0.7 | No clear effect |
| remove LVIS rare labels from training data (ablation) | OWL-ViT R26+B/32 (LVIS) · LVIS rare | APr: 0.2 | No clear effect |
Closest papers by meaning
- Language-conditioned Detection Transformer (CVPR 2024 · similarity 0.92)
- Generative Region-Language Pretraining for Open-Ended Object Detection (CVPR 2024 · similarity 0.91)
- How to Evaluate the Generalization of Detection? A Benchmark for Comprehensive Open-Vocabulary Detection (AAAI 2024 · similarity 0.90)
- Simple Image-Level Classification Improves Open-Vocabulary Object Detection (AAAI 2024 · similarity 0.90)
- Taming Self-Training for Open-Vocabulary Object Detection (CVPR 2024 · similarity 0.90)
- DitHub: A Modular Framework for Incremental Open-Vocabulary Object Detection (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
