Research map / Vision-language and multimodal / Multimodal LLMs
Cheap and Quick: Efficient Vision-Language Instruction Tuning for Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add vision modality to mixture-of-modality training (ablation) | LaVIN text-only baseline · ScienceQA test | average accuracy: 82.65 -> 86.32 | Helped |
| jointly optimize image encoder and LLM (ablation) | LaVIN-7B · ScienceQA test | average accuracy: 86.32 -> 87.34 | Helped |
| use stronger image encoder (ViT-L/14) (ablation) | LaVIN-7B · ScienceQA test | average accuracy: 87.34 -> 88.33 | Helped |
| add MM-Adapter modality routing (ablation) | LaVIN-7B · ScienceQA test | average accuracy: 88.33 -> 89.41 | Helped |
| scale LLM from 7B to 13B (ablation) | LaVIN · ScienceQA test | average accuracy: 89.41 -> 90.50 | Helped |
| train 40 epochs instead of 20 | LaVIN-13B · ScienceQA test | average accuracy: 90.50 -> 90.83 | Helped |
| add 0.6M pre-training data | LaVIN-13B · COCO Captioning | CIDEr: 126.9 -> 131.7 | Helped |
| use MMA adapters instead of full LLM fine-tuning (LLaVA) | LaVIN-7B vs LLaVA · ScienceQA training cost | training time: 7 hours -> 1.4 hours | Helped |
| apply MMA to LLaMA (modality routing adapters) | LaVIN vs LLaMA-Base · TruthfulQA | Mc1 accuracy: 38.7 -> 47.9 | Helped |
Closest papers by meaning
- VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks (NeurIPS 2024 · similarity 0.90)
- CuMo: Scaling Multimodal LLM with Co-Upcycled Mixture-of-Experts (NeurIPS 2024 · similarity 0.90)
- MoDA: Modulation Adapter for Fine-Grained Visual Understanding in Instructional MLLMs (ICML 2026 · similarity 0.90)
- Towards General Continuous Memory for Vision-Language Models (NeurIPS 2025 · similarity 0.90)
- Tackling Vision Language Tasks through Learning Inner Monologues (AAAI 2024 · similarity 0.89)
- LRM-LLaVA: Overcoming the Modality Gap of Multilingual Large Language-Vision Model for Low-Resource Languages (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.