Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Seeing and Knowing in the Wild: Open-domain Visual Entity Recognition with Large-scale Knowledge Graphs via Contrastive Learning
Hongkuan Zhou, Lavdim Halilaj, Sebastian Monka, Stefan Schmid, Yuqicheng Zhu, Jingcheng Wu, Nadeem Nazer, Steffen Staab
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add lead images of entities (ablation) | KnowCoL-L · OVEN entity test | HM: 30.0 -> 31.5 | Improved |
| add KG hierarchical knowledge (ablation) | KnowCoL-L · OVEN entity test | unseen accuracy: 25.3 -> 29.1 | Improved |
| use simple addition instead of 2-layer MLP fusion (ablation) | KnowCoL-L · OVEN entity test | HM: 28.7 -> 31.5 | Improved |
| use transformer encoder fusion instead of addition (ablation) | KnowCoL-L · OVEN entity test | seen vs unseen accuracy: seen 34.3 -> 54.4; unseen 29.1 -> 16.6 | Mixed |
| add local image patch tokens to transformer fusion (ablation) | KnowCoL-L · OVEN entity test | unseen accuracy: 16.6 -> 17.9 | Improved |
| use TransE with cosine instead of Euclidean distance (ablation) | KnowCoL-L · OVEN entity test | HM: 31.0 -> 31.5 | Improved |
| use TransE instead of DistMult (ablation) | KnowCoL-L · OVEN entity test | HM: 30.9 -> 31.5 | Improved |
| scale CLIP backbone from ViT-L to ViT-bigG (ablation) | KnowCoL · OVEN entity test | HM: 31.5 -> 38.8 | Improved |
| add knowledge-guided contrastive learning on same ViT-L-14 | CLIP2CLIP · OVEN entity test | HM: 11.4 -> 31.5 | Improved |
| KnowCoL-bigG vs Auto-VER-14B on seen entities | Auto-VER-14B · OVEN entity test | seen accuracy: 65.0 -> 41.8 | Hurt |
| KnowCoL-bigG vs Auto-VER-14B on unseen entities | Auto-VER-14B · OVEN entity test | unseen accuracy: 18.6 -> 36.1 | Improved |
Closest papers by meaning
- Web-Scale Visual Entity Recognition: An LLM-Driven Data Approach (NeurIPS 2024 · similarity 0.89)
- Reverse Region-to-Entity Annotation for Pixel-Level Visual Entity Linking (AAAI 2025 · similarity 0.88)
- A Generative Approach for Wikipedia-Scale Visual Entity Recognition (CVPR 2024 · similarity 0.87)
- MCL-NER: Cross-Lingual Named Entity Recognition via Multi-View Contrastive Learning (AAAI 2024 · similarity 0.87)
- A Dual-Way Enhanced Framework from Text Matching Point of View for Multimodal Entity Linking (AAAI 2024 · similarity 0.86)
- Cyclic Contrastive Knowledge Transfer for Open-Vocabulary Object Detection (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
