Research map / Vision-language and multimodal / Multimodal LLMs
Open-Set Recognition in the Age of Vision-Language Models
Dimity Miller, Niko Suenderhauf, Alex Kenna, Keita Mason
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| evaluate VLM classifiers with a finite query set that omits the image's true class | CLIP, ALIGN, CoCa, ImageBind, SigLIP, LanguageBind · ImageNet-1k val | precision at 95% recall: precision between 46.2% and 58.4% | Hurt |
| use softmax or entropy instead of cosine similarity as uncertainty (ablation) | VLM classifiers · ImageNet-1k val | AuPR: 61.3 -> 71.5 (CLIP cosine vs softmax) | Improved |
| add 2500 random-word negative queries (ablation) | ImageBind · ImageNet-1k val | AuPR / accuracy: AuPR 79.1 -> 84.6; accuracy 76.8 -> 68.4 | Mixed |
| add 500 random-word negative queries (ablation) | CLIP · ImageNet-1k val | AuPR: 71.5 -> 73.4 | Improved |
| add random-embedding negatives (ablation) | SigLIP · ImageNet-1k val | accuracy: 75.7 -> 21.1 | Hurt |
| add 100 random-word negative queries (ablation) | OV-DETR · COCO val | AuPR: 75.8 -> 84.7 | Improved |
| add random-embedding negatives to RegionCLIP (ablation) | RegionCLIP · COCO val | mAP@0.5: 44.6 -> 1.6 | Hurt |
| add random-word or random-embedding negatives (ablation) | ViLD and CORA · COCO val | AuPR: 60.1 -> 55.0 (ViLD +100 words) | Hurt |
| increase query set size with related WordNet classes (ablation) | VLM classifiers · ImageNet-1k val | closed-set and open-set performance | Hurt |
| use AuROC instead of AuPR for detectors | COCO detectors · COCO val | metric reliability | Hurt |
Closest papers by meaning
- From Coarse to Fine-Grained Open-Set Recognition (CVPR 2024 · similarity 0.88)
- Seeing Clearly, Reasoning Confidently: Plug-and-Play Remedies for Vision Language Model Blindness (CVPR 2026 · similarity 0.88)
- VL-SAM-V2: Open-World Object Detection with General and Specific Query Fusion (NeurIPS 2025 · similarity 0.88)
- The Neglected Tails in Vision-Language Models (CVPR 2024 · similarity 0.87)
- Synthesize Diagnose and Optimize: Towards Fine-Grained Vision-Language Understanding (CVPR 2024 · similarity 0.87)
- UniBench: Visual Reasoning Requires Rethinking Vision-Language Beyond Scaling (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
