Research map / Vision-language and multimodal / CLIP and zero-shot
K-LITE: Learning Transferable Visual Models with External Knowledge
Sheng Shen, Chunyuan Li, Xiaowei Hu, Yujia Xie, Jianwei Yang, Pengchuan Zhang, Zhe Gan, Lijuan Wang, Lu Yuan, Ce Liu, Kurt Keutzer, Trevor Darrell and 2 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Wiktionary definition knowledge in pre-training and evaluation | UniCL · ImageNet-1K | zero-shot accuracy: 28.16 -> 30.23 | Improved |
| Add Wiktionary definition knowledge in pre-training and evaluation | UniCL · ICinW (20 datasets) | zero-shot score: 27.15 -> 33.44 | Improved |
| Add Wiktionary definition knowledge | UniCL · ImageNet-1K | zero-shot accuracy: 83.74% to 85.80% | Improved |
| Continue pre-training without knowledge rather than with knowledge (ablation) | Florence · ImageNet-1K | zero-shot accuracy: 85.35% vs 85.80% | Hurt |
| Use Wiktionary definitions instead of no external knowledge | GLIP · LVIS | box mAP: 18.5 -> 21.3 | Improved |
| Use Wiktionary definitions instead of no external knowledge | GLIP · ODinW (13 datasets) | box mAP: 27.5 -> 31.7 | Improved |
| Use WordNet hierarchy instead of Wiktionary definitions (ablation) | K-LITE image classifier · ImageNet-1K | zero-shot accuracy: 30.23 -> 29.03 | Hurt |
| Use WordNet definitions instead of Wiktionary definitions (ablation) | K-LITE image classifier · ICinW (20 datasets) | zero-shot score: 33.44 -> 29.14 | Hurt |
| Use Wiktionary definitions instead of no knowledge (ablation) | UniCL · ImageNet-1K | zero-shot accuracy: 28.16 -> 4.93 when knowledge is added to the baseline at test time | Improved |
| Remove knowledge at evaluation after knowledge-augmented training (ablation) | K-LITE · ICinW (20 datasets) | zero-shot score: 33.44 -> 29.03 | Hurt |
| Use two selective language branches instead of only the knowledge branch (ablation) | K-LITE modularized model · ImageNet-1K | zero-shot accuracy: 28.40% to 28.90% | Improved |
| Use the caption combine scheme rather than the default concat scheme (ablation) | K-LITE · ImageNet-1K | zero-shot accuracy: 36.67 -> 42.36 | Improved |
Closest papers by meaning
- Hawaii: Hierarchical Visual Knowledge Transfer for Efficient Vision-Language Models (NeurIPS 2025 · similarity 0.88)
- UKnow: A Unified Knowledge Protocol with Multimodal Knowledge Graph Datasets for Reasoning and Vision-Language Pre-Training (NeurIPS 2024 · similarity 0.87)
- Vision‑Language‑Vision Auto‑Encoder: Scalable Knowledge Distillation from Diffusion Models (NeurIPS 2025 · similarity 0.87)
- MoVE-KD: Knowledge Distillation for VLMs with Mixture of Visual Encoders (CVPR 2025 · similarity 0.87)
- InstructCV: Instruction-Tuned Text-to-Image Diffusion Models as Vision Generalists (ICLR 2024 · similarity 0.87)
- Cyclic Contrastive Knowledge Transfer for Open-Vocabulary Object Detection (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
