Research map / Vision-language and multimodal / CLIP and zero-shot
Knowledge Transfer from Interaction Learning
Yilin Gao, Kangyi Chen, Zhongxing Peng, Hengjie Lu, Shugong Xu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add interaction queries supervised by VLM interactions with gated control to ViT-L | ViT-L/16 · TinyImageNet | top-1 accuracy: 87.83 -> 91.08 | Improved |
| add interaction queries to the supervised I-ViT (ablation) | I-ViT-L/16 · TinyImageNet | top-1 accuracy: 80.7 -> 91.1 | Improved |
| add gated control network to fuse interactions (ablation) | I-ViT-L/16 · TinyImageNet | top-1 accuracy: 90.3 -> 91.1 | Improved |
| add VLM interaction constraint (supervision of C_VLM) on interaction queries (ablation) | I-ViT-L/16 · TinyImageNet | top-1 accuracy: 89.6 -> 91.1 | Improved |
| supervise original queries directly with VLM interactions (ablation) | ViT-L/16 · TinyImageNet | top-1 accuracy: 87.8 -> 80.7 | Hurt |
| add VLM interactions on limited training data | ViT-L/16 from scratch · TinyImageNet 10% data | top-1 accuracy: 12.64 -> 15.60 | Improved |
| add VLM interactions to DETR | DETR · COCO 2017 detection | mAP: 42.0 -> 43.6 | Improved |
| add VLM interactions to DETR segmentation | DETR · COCO 2017 segmentation | AP mask: 31.1 -> 33.5 | Improved |
| add visual bounding-box prompts when extracting VLM interactions (ablation) | VLM interaction extraction · COCO val2017 | mAP: 41.6 -> 43.6 | Improved |
| extract VLM interaction with foreground-background prompt instead of description prompt (ablation) | I-ViT-S/16 · TinyImageNet | top-1 accuracy: 80.59 -> 81.52 | Improved |
| add VLM interactions on sketch domain | ViT-L/16 · PACS Sketch zero-shot | accuracy: 15.49 -> 15.18 | No clear effect |
| add VLM interactions to small object detection | Conditional-DETR · VOC few-shot | AP_S: 5.0 -> 4.9 | No clear effect |
Closest papers by meaning
- Hawaii: Hierarchical Visual Knowledge Transfer for Efficient Vision-Language Models (NeurIPS 2025 · similarity 0.88)
- Measuring Cross-Modal Synergy: A Benchmark for VLM Explainability (NeurIPS 2026 · similarity 0.88)
- UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm (CVPR 2026 · similarity 0.88)
- Pathways of Visual Information Flow in Vision-Language Models (NeurIPS 2026 · similarity 0.88)
- VGR: Visual Grounded Reasoning (ICLR 2026 · similarity 0.88)
- Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
