Research map / Vision-language and multimodal / Multimodal LLMs

SpineBench: A Clinically Salient, Level-Aware Benchmark Powered by the SpineMed-450k Corpus

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces SpineMed-450k, a ~450k-instance clinician-curated multimodal instruction corpus for spine diagnosis, plus the SpineBench evaluation, and fine-tunes Qwen2.5-VL-7B into SpineGPT, which scores 87.44% average on SpineBench.

Problem

AI-assisted spine diagnosis lacks traceable, clinically grounded multimodal instruction data and standardized spine-specific benchmarks for level-aware reasoning across X-ray, CT, and MRI.

Approach

A clinician-in-the-loop pipeline builds a large spine instruction corpus from textbooks, guidelines, open spine datasets and ~1,000 de-identified hospital cases, using a two-stage LLM draft-and-revision generation with provenance tracking; a clinician-validated benchmark (SpineBench) with rubric-based report scoring evaluates LVLMs, and a 7B model is curriculum-tuned on the corpus.

Key results

• SpineGPT (Qwen2.5-VL-7B fine-tuned) reaches 87.44% average on SpineBench, above all open-source models evaluated (best open-source GLM-4.5V at 83.26%) but below Gemini-2.5-Pro at 89.23%.
• Ablation: training only on the spine subset gives 87.07 average vs 87.89 for the full curriculum; general medical data alone drops performance to 65.31 from a 74.95 baseline.
• Qwen2.5-VL-7B baseline scores 74.95 close-ended average; text-image gap of 4.36 to 7.44 points is observed across models.

Surprising findings

• General medical data alone (PubMedVision and reasoning sets) degraded SpineBench performance from 74.95 to 65.31.
• Medgemma-27B, a medical-specialized model, scored below the 7B SpineGPT despite being about 4 times larger.

Limitations

• Hospital data cannot be fully released; only a de-identified subset under a DUA is planned.
• Proprietary models GPT-4 and Gemini are not directly compared in the current version, and the model is limited to 7B parameters.
• SpineGPT trails Gemini-2.5-Pro (87.44 vs 89.23) on the average score.

Experimental setup

Datasets: SpineMed-450k PubMedVision medical-o1-reasoning-SFT Medical-R1-Distill-Data MedThoughts-8K VerSe Spark Europe PMC

Benchmarks: SpineBench

Metrics: accuracy rubric score (1-5 per dimension, normalized to 100)

Baselines: GPT-5 O3 Gemini-2.5-Pro Claude4 GPT-4o GPT-5-mini Gemini-2.5-Flash GLM-4.5V Qwen2.5-VL-72B Linshu-32B Medgemma-27B HuatuoGPT-7B

Builds on: Qwen2.5-VL PubMedVision ms-swift

Closest papers by meaning

• Beyond Classification Accuracy: Neural-MedBench and the Need for Deeper Reasoning Benchmarks (ICLR 2026 · similarity 0.86)
• RAD: Towards Trustworthy Retrieval-Augmented Multi-modal Clinical Diagnosis (NeurIPS 2025 · similarity 0.85)
• M3CoTBench: Benchmark Chain-of-Thought of MLLMs in Medical Image Understanding (ICLR 2026 · similarity 0.84)
• CXReasonBench: A Benchmark for Evaluating Structured Diagnostic Reasoning in Chest X-rays (NeurIPS 2025 · similarity 0.84)
• OmniBrainBench: A Comprehensive Multimodal Benchmark for Brain Imaging Analysis Across Multi-stage Clinical Tasks (CVPR 2026 · similarity 0.84)
• Large-scale and Fine-grained Vision-language Pre-training for Enhanced CT Image Understanding (ICLR 2025 · similarity 0.84)