Research map / Vision-language and multimodal / Multimodal LLMs

Gastric-X: A Multimodal Multi-Phase Benchmark Dataset for Advancing Vision-Language Models in Gastric Cancer Analysis

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Gastric-X, a 1.7K-case multimodal gastric cancer benchmark (multi-phase CT, endoscopy, biochemistry, reports, bounding boxes, VQA pairs) and evaluates adapted VLMs on five tasks.

Problem

Medical VLM research lacks structured multimodal datasets that combine multi-phase 3D CT, endoscopic images, lab indicators, and clinical reports for gastric cancer diagnosis.

Approach

A patient-level gastric cancer dataset aligning four-phase CT, endoscopy, structured biochemistry/EHR, expert 3D bounding boxes, reports and LLM-generated clinician-verified VQA pairs, plus a benchmark that adapts general and medical VLMs to 3D multi-phase input and tests adding bounding-box overlays and abnormal lab tables.

Key results

• X2-VLM-Med reaches 91.5% AUC on VQA with Image+Table+BBox, up from 85.3% with image only.
• X2-VLM-Med achieves BERTScore F1 of 82.0 on report generation with all inputs, versus 68.7 image-only.
• X2-VLM-Med attains R@1 of 48.9% (Image→Text) for cross-modal retrieval, the best among evaluated models.

Limitations

• Dataset is planned for release only after acceptance and under access-controlled consent; full data not yet public.
• Several VLM baselines were adapted with ad hoc 3D modifications (Swin encoder replacement, multi-channel concatenation) that may not reflect their native capability.
• Reported Gastric-X results come from a single-institution clinical dataset with limited external validation.

Experimental setup

Datasets: Gastric-X

Benchmarks: Gastric-X

Metrics: Precision Accuracy F1 AUC ROUGE-L BLEU-4 METEOR BERTScore F1 Recall@K MedR

Baselines: LLaVA-1.5-7B LLaVA-Med v1.5 Med-Flamingo BLIP-2 MedVInT ResNet-50 Swin Transformer Faster R-CNN

Builds on: X2-VLM LLaVA-Med Med-Flamingo MedVInT

Closest papers by meaning

• EndoBench: A Comprehensive Evaluation of Multi-Modal Large Language Models for Endoscopy Analysis (NeurIPS 2025 · similarity 0.88)
• PathView-Bench: Can Multimodal Large Language Models Achieve Fine-grained Multiscale Understanding of Pathology Images? (NeurIPS 2026 · similarity 0.87)
• TumorChain: Interleaved Multimodal Chain-of-Thought Reasoning for Traceable Clinical Tumor Analysis (ICLR 2026 · similarity 0.87)
• Toward a Vision-Language Foundation Model for Medical Data: Multimodal Dataset and Benchmarks for Vietnamese PET/CT Report Generation (NeurIPS 2025 · similarity 0.87)
• M3CoTBench: Benchmark Chain-of-Thought of MLLMs in Medical Image Understanding (ICLR 2026 · similarity 0.86)
• CXReasonBench: A Benchmark for Evaluating Structured Diagnostic Reasoning in Chest X-rays (NeurIPS 2025 · similarity 0.86)