AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

SQ-LLaVA: Self-Questioning for Large Vision-Language Assistant

Guohao Sun, Can Qin, JIAMINAN WANG, Zeyuan Chen, Ran Xu, Zhiqiang Tao

ECCV 2024 · Vision-language and multimodal › Multimodal LLMs · approach: Visual instruction tuning · 14 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add visual self-questioning as extra training task (ablation)SQ-LLaVA 7B · 5 VQA benchmarks, 558K PT / 665K ITaverage accuracy: 66.4 -> 67.5Improved
add prototype extractor (ablation)SQ-LLaVA 7B · 5 VQA benchmarks, 558K PT / 665K ITaverage accuracy: 66.9 -> 67.5Improved
add LoRA on the vision encoder (ablation)SQ-LLaVA 7B · 5 VQA benchmarks, 558K PT / 665K ITaverage accuracy: 67.3 -> 67.5Improved
add prototype extractor on larger data scale (ablation)SQ-LLaVA 7B · 5 VQA benchmarks, 1200K PT / 700K ITaverage accuracy: 69.6 -> 69.6No clear effect
add all three components versus baseline (ablation)SQ-LLaVA 7B · 5 VQA benchmarks, 558K PT / 665K ITaverage accuracy: 65.9 -> 67.5Improved
add all three components on larger data (ablation)SQ-LLaVA 7B · 5 VQA benchmarks, 1200K PT / 700K ITaverage accuracy: 67.6 -> 69.6Improved
SQ-LLaVA versus LLaVA-v1.5 on LLaVA-BenchSQ-LLaVA 7B · LLaVA-Bench (in the wild)accuracy: 63.4 -> 66.3Improved
SQ-LLaVA versus LLaVA-v1.5 on POPE hallucinationSQ-LLaVA 7B · POPEaccuracy: 85.9 -> 87.7Improved
SQ-LLaVA versus ShareGPT4V on VQAv2, VizWiz, MM-VetSQ-LLaVA 7B · VQAv2 / VizWiz / MM-Vetaccuracy: MM-Vet 37.6 vs 32.5; VizWiz 57.2 vs 54.0Mixed
SQ-LLaVA zero-shot COCO captioning versus LLaVA-v1.5SQ-LLaVA · COCO zero-shot captioningCIDEr: 111.46 -> 110.77No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy