Research map / Vision-language and multimodal / Multimodal LLMs
LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Stage 2 biomedical instruction-tuning after Stage 1 concept alignment (ablation) | LLaVA · Biomedical visual chatbot (overall) | relative score: 23.3 -> 50.2 | Helped |
| Increase instruction-following data from 10K to 60K samples (ablation) | LLaVA-Med · Biomedical visual chatbot (overall) | relative score: 39.9 -> 49.4 | Helped |
| Add inline mentions as context when generating instruction data (ablation) | LLaVA-Med 60K · Biomedical visual chatbot (overall) | relative score: 49.4 -> 50.2 | Helped |
| Replace the full two-stage adaptation with Stage 1 concept alignment alone (ablation) | LLaVA-Med · Biomedical visual chatbot (overall) | relative score: 50.2 -> 23.3 | Hurt |
| Increase Stage 1 training from 1 to 3 epochs (ablation) | LLaVA-Med without Stage 2 or downstream fine-tuning · VQA-RAD zero-shot open | recall: 15.27 -> 15.33 | Helped |
| Add biomedical domain adaptation and instruction-tuning | LLaVA · VQA-RAD test open | recall: 50.00 -> 61.52 | Helped |
| Use BioMed CLIP rather than general-domain CLIP as the vision encoder initialization (ablation) | LLaVA-Med · SLAKE test open | recall: 83.08 -> 87.11 | Helped |
| Increase language model size from 7B to 13B (ablation) | LLaVA-Med 60K-IM · PathVQA zero-shot open | recall: 37.59 -> 38.82 | Helped |
| Increase downstream fine-tuning to 9 epochs (ablation) | LLaVA-Med 60K-IM, Stage 1: 1 epoch and Stage 2: 3 epochs · VQA-RAD zero-shot open | recall: 28.23 -> 66.26 | Helped |
| Fine-tune the LLaVA-Med model on each downstream dataset (ablation) | LLaVA-Med (From LLaVA) · VQA-RAD test closed | accuracy: 80.88 -> 84.19 | Helped |
| Use Vicuna rather than LLaVA language-model initialization (ablation) | LLaVA-Med · VQA-RAD test open | recall: 61.52 -> 64.39 | Helped |
Closest papers by meaning
- Towards a Multimodal Large Language Model with Pixel-Level Insight for Biomedicine (AAAI 2025 · similarity 0.92)
- MedMax: Mixed-Modal Instruction Tuning for Training Biomedical Assistants (NeurIPS 2025 · similarity 0.91)
- LLaDA-MedV: Exploring Large Language Diffusion Models for Biomedical Image Understanding (CVPR 2026 · similarity 0.91)
- GMAI-VL & GMAI-VL-5.5M: A Large Vision-Language Model and a Comprehensive Multimodal Dataset Towards General Medical AI (AAAI 2026 · similarity 0.89)
- OpenMedReason: Scientific Reasoning Supervision for Medical Vision–Language Models (NeurIPS 2026 · similarity 0.89)
- MIMO: A Medical Vision Language Model with Visual Referring Multimodal Input and Pixel Grounding Multimodal Output (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.