Research map / Vision-language and multimodal / CLIP and zero-shot
Vocabulary-free Image Classification
Alessandro Conti, Enrico Fini, Massimiliano Mancini, Paolo Rota, Yiming Wang, Elisa Ricci
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CaSED retrieval and candidate scoring rather than BLIP-2 VQA | VIC classification · Ten-dataset average | Cluster Accuracy: 43.1 vs 38.7; +4.4% | Improved |
| Use CaSED rather than BLIP-2 VQA | VIC classification · Ten-dataset average | Semantic IoU: 17.7 vs 16.0; +1.7% | Improved |
| Use combined visual and language candidate scoring (ablation) | CaSED · Ten-dataset average | Cluster Accuracy: 41.7 -> 43.1; +1.4% | Improved |
| Use language-only candidate scoring instead of visual-only scoring (ablation) | CaSED · Ten-dataset average | Cluster Accuracy: 41.7 -> 42.7; +1% | Improved |
| Generate candidates from retrieved captions rather than BLIP-2 generation (ablation) | CaSED with visual scoring · Ten-dataset average | Cluster Accuracy: 23.3 -> 41.7 | Improved |
| Use CC12M alone rather than the combined database (ablation) | CaSED · Ten-dataset average | Cluster Accuracy: 43.1 -> 44.0 | Improved |
| Use CC3M rather than the combined database (ablation) | CaSED · Ten-dataset average | Cluster Accuracy: 34.2 vs 43.1; +8.9% over CC3M | Hurt |
| Retrieve 10 captions rather than 1 (ablation) | CaSED · Ten-dataset average | Cluster Accuracy: 30.9 -> 43.1 | Improved |
| Retrieve 20 captions rather than 10 (ablation) | CaSED · Ten-dataset average | Semantic Similarity: 50.4 -> 49.3 | Hurt |
| Add candidate removal operations (ablation) | CaSED candidate filtering · Ten-dataset average | Cluster Accuracy: 27.7 -> 34.1; +6.4% | Improved |
| Add candidate standardization after removal (ablation) | CaSED candidate filtering · Ten-dataset average | Cluster Accuracy: 34.1 -> 41.3; +7.2% | Improved |
| Add final POS and occurrence filtering (ablation) | CaSED candidate filtering · Ten-dataset average | Cluster Accuracy: 41.3 -> 43.1 | Improved |
Closest papers by meaning
- Vocabulary-Free Image Classification and Semantic Segmentation (TPAMI 2026 · similarity 0.96)
- Visual Classification via Description from Large Language Models (ICLR 2023 · similarity 0.91)
- OVMR: Open-Vocabulary Recognition with Multi-Modal References (CVPR 2024 · similarity 0.89)
- Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Models (CVPR 2024 · similarity 0.89)
- Follow-Up Differential Descriptions: Language Models Resolve Ambiguities for Image Classification (ICLR 2024 · similarity 0.89)
- Training-free Open-Vocabulary Semantic Segmentation via Diverse Prototype Construction and Sub-region Matching (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
