Research map / Label-efficient and robust learning / Recognition and long-tail
Towards Open-Ended Visual Recognition with Large Language Models
Qihang Yu, Xiaohui Shen, Liang-Chieh Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| raise input resolution from 224 to 448 (ablation) | OSM · six-dataset mask classification | Avg accuracy: 57.3 -> 70.1 (+12.8%) | Improved |
| raise input resolution beyond 1120 (ablation) | OSM · six-dataset mask classification | Avg accuracy: 79.9 -> 76.6 (1344), 75.9 (1568) | Hurt |
| use sliding-window frozen CLIP-ViT instead of global high-res input (ablation) | OSM · six-dataset mask classification | Avg accuracy: 71.5 -> 78.7 | Improved |
| use overlapping sliding windows (stride 112) (ablation) | OSM · six-dataset mask classification | Avg accuracy: 78.7 -> 79.6 | No clear effect |
| add vocabulary-specific Mode Query (ablation) | OSM · six-dataset mask classification | Avg accuracy: 74.8 -> 78.7; NIV 4.45 -> 0.61 | Improved |
| restrict context queries to enlarged bounding box (0.5x) instead of whole image (ablation) | OSM MaskQ-Former · six-dataset mask classification | Avg accuracy: 78.7 -> 81.3 | Improved |
| replace frozen LLM generative predictor with learnable discriminative embeddings (ablation) | OSM · six-dataset mask classification | Avg accuracy: 78.7 vs 78.9 | No clear effect |
| use OSM instead of LMSeg for mask classification with ResNet50 proposals | kMaX-DeepLab proposals · COCO PQ | PQ: 38.6 -> 53.3 | Improved |
| add geometric ensemble with frozen CLIP (ablation) | OSM zero-shot · ADE20K zero-shot | PQ: 21.4 -> 26.9 | Improved |
Closest papers by meaning
- OVMR: Open-Vocabulary Recognition with Multi-Modal References (CVPR 2024 · similarity 0.89)
- Open-Det: An Efficient Learning Framework for Open-Ended Detection (ICML 2025 · similarity 0.87)
- Boosting Segment Anything Model Towards Open-Vocabulary Learning (AAAI 2025 · similarity 0.87)
- ProxyDet: Synthesizing Proxy Novel Classes via Classwise Mixup for Open-Vocabulary Object Detection (AAAI 2024 · similarity 0.87)
- Overcoming the Pitfalls of Vision-Language Model Finetuning for OOD Generalization (ICLR 2024 · similarity 0.87)
- How to Evaluate the Generalization of Detection? A Benchmark for Comprehensive Open-Vocabulary Detection (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
