Research map / Vision-language and multimodal / Human and embodied modeling
Localized Symbolic Knowledge Distillation for Visual Commonsense Models
Jae Sung Park, Jack Hessel, Khyathi Chandu, Paul Pu Liang, Ximing Lu, Peter West, Youngjae Yu, Qiuyuan Huang, Jianfeng Gao, Ali Farhadi, Yejin Choi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Distill localized commonsense knowledge into BLIP-2 ViT-G | BLIP-2 ViT-G · VCR Q→A | accuracy: 56.1 -> 59.0 | Improved |
| Distill localized commonsense knowledge into BLIP-2 ViT-G | BLIP-2 ViT-G · VCR QA→R | accuracy: 49.8 -> 56.4 | Improved |
| Distill localized commonsense knowledge into BLIP-2 ViT-G | BLIP-2 ViT-G · VCR Q→AR | accuracy: 28.0 -> 33.4 | Improved |
| Distill localized commonsense knowledge into BLIP-2 ViT-G | BLIP-2 ViT-G · Sherlock | Comparison: 19.5 -> 29.7 | Improved |
| Distill localized commonsense knowledge into BLIP-2 ViT-G | BLIP-2 ViT-G · VisualCOMET | Acc@50: 39.0 -> 40.3 | Improved |
| Distill localized commonsense knowledge into BLIP-2 ViT-G | BLIP-2 ViT-G · AOKVQA | multiple-choice accuracy: 68.0 -> 68.9 | Improved |
| Distill localized commonsense knowledge into BLIP-2 ViT-G | BLIP-2 ViT-G · SNLI-VE | classification accuracy: 33.4 -> 40.3 | Improved |
| Distill localized commonsense knowledge into BLIP-2 ViT-G | BLIP-2 ViT-G · Visual7W Telling QA | accuracy: 77.1 -> 79.5 | Improved |
| Scale the filtered LSKD corpus from 150K to 1M examples (ablation) | BLIP-2 ViT-G + LSKD · Sherlock | Comparison: 28.6 -> 29.7 | Improved |
| Scale the filtered LSKD corpus from 150K to 1M examples (ablation) | BLIP-2 ViT-G + LSKD · SNLI-VE | classification accuracy: 38.0 -> 40.3 | Improved |
| Add QA descriptors to the full verbalization used for knowledge generation (ablation) | LSKD corpus generation · 500-image descriptor ablation | average critic score: 49.0 to 58.4 | Improved |
| Filter generated statements with the supervised critic rather than keeping unfiltered statements (ablation) | LSKD corpus generation · localized visual reasoning tasks | downstream accuracy | Improved |
Closest papers by meaning
- Activating Visual Context and Commonsense Reasoning Through Masked Prediction in VLMs (AAAI 2026 · similarity 0.90)
- LION: Empowering Multimodal Large Language Model with Dual-Level Visual Knowledge (CVPR 2024 · similarity 0.90)
- Towards General Continuous Memory for Vision-Language Models (NeurIPS 2025 · similarity 0.89)
- VLM-R³: Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thought (NeurIPS 2025 · similarity 0.89)
- The Narrow Gate: Localized Image-Text Communication in Native Multimodal Models (NeurIPS 2025 · similarity 0.88)
- Benchmarking Multimodal Mathematical Reasoning with Explicit Visual Dependency (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
