Research map / Vision-language and multimodal / Multimodal LLMs
Lingshu: Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning
Weiwen Xu, Hou Pong Chan, Long Li, Mahani Aljunied, Ruifeng Yuan, Jianyu Wang, Chenghao Xiao, 陈贵珍, Chaoqun Liu, Zhaodonghui Li, Yu Sun, Junao Shen and 8 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add medical text data (173K samples) to instruction tuning (ablation) | Lingshu-Med-7B · MMMU-Med | accuracy: 50.9 -> 54.0 | Improved |
| add medical text data to instruction tuning (ablation) | Lingshu-Med-7B · MedXpertQA | accuracy: 24.0 -> 26.7 | Improved |
| add medical multimodal data (2.7M) to instruction tuning (ablation) | Lingshu-Med-7B · SLAKE | accuracy: 69.0 -> 83.1 | Improved |
| add medical multimodal data to instruction tuning (ablation) | Lingshu-Med-7B · PathVQA | accuracy: 45.2 -> 61.9 | Improved |
| add general multimodal data (1.2M) to instruction tuning (ablation) | Lingshu-Med-7B · MMMU-Med | accuracy: 51.4 -> 54.0 | Improved |
| add general text data (1M) to instruction tuning (ablation) | Lingshu-Med-7B · VQA-RAD | accuracy: 65.2 -> 67.9 | Improved |
| add synthetic multimodal medical data (1.1M) (ablation) | Lingshu-Med-7B · OmniMedVQA | accuracy: 71.6 -> 82.9 | Improved |
| add distilled medical CoT text (162K) (ablation) | Lingshu-Med-7B · PMC-VQA | accuracy: 53.6 -> 56.3 | Improved |
| add Stage 3 medical captions (2M) (ablation) | Lingshu-Med-7B · MMMU-Med | accuracy: 52.9 -> 54.0 | Improved |
| add Stage 1&2 caption alignment (4.8M captions) (ablation) | Lingshu-Med-7B · PMC-VQA | accuracy: 54.9 -> 56.3 | Improved |
| add alignment stages before Stage 3 vs direct fine-tuning of Qwen2.5-VL on Stage 3 data (ablation) | Lingshu-Med-7B · medical benchmarks average | accuracy | Improved |
| apply RLVR (GRPO) on top of Lingshu-Med-7B | Lingshu-Med-7B · medical multimodal avg (7 benchmarks) | accuracy: 61.8 -> 61.5 | No clear effect |
Closest papers by meaning
- Chiron-o1: Igniting Multimodal Large Language Models towards Generalizable Medical Reasoning via Mentor-Intern Collaborative Search (NeurIPS 2025 · similarity 0.91)
- MedMO: Grounding and Understanding Multimodal Large Language Model for Medical Images (CVPR 2026 · similarity 0.91)
- Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoning (CVPR 2026 · similarity 0.91)
- Interpretable Bilingual Multimodal Large Language Model for Diverse Biomedical Tasks (ICLR 2025 · similarity 0.91)
- OctoMed: Data Recipes for State-of-the-Art Multimodal Medical Reasoning (CVPR 2026 · similarity 0.91)
- GMAI-VL & GMAI-VL-5.5M: A Large Vision-Language Model and a Comprehensive Multimodal Dataset Towards General Medical AI (AAAI 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
