Research map / Vision-language and multimodal / Multimodal LLMs

TumorChain: Interleaved Multimodal Chain-of-Thought Reasoning for Traceable Clinical Tumor Analysis

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces TumorChain, an interleaved multimodal chain-of-thought framework with a segmentation expert and abnormality classifier for traceable 3D CT tumor analysis, plus the 1.5M-sample TumorCoT-1.5M dataset and a CoT evaluation protocol.

Problem

Med-LVLMs struggle to link 3D CT radiology findings to impressions and pathology-level conclusions (TNM staging, nodal/metastasis status) with traceable, multi-step reasoning, and tumor-specific multimodal datasets are scarce.

Approach

Iteratively interleave LLM reasoning with organ-level ROI tokens: the LLM proposes organs, a segmentation expert returns masks whose local 3D tokens are fed back for the next reasoning round, and a lightweight normal/abnormal classifier on local organ tokens is jointly trained to sharpen visual discrimination.

Key results

• TumorChain-7B reaches 84.41% average accuracy on the TumorCoT test split, versus 45.98% for the best open baseline (MiniCPM-V4.5-9B) shown in Table 1 and 51.59% for GPT-5-Mini.
• On the public DeepTumorVQA benchmark, TumorChain-7B averages 57.51%, 14.84 points above the second-best MedVLM-R1 (42.67%), without training on that dataset.
• TumorChain-7B attains CoT_e 58.33 versus 61.23 for GPT-5-Mini, the best CoT_e score in the table.

Surprising findings

• Off-the-shelf 3D medical models (RadFM, M3D-Phi-3) score below many 2D and general models, and some 3D models underperform 2D models on CT analysis.
• Ablation on the classification loss weight shows that both no auxiliary loss (alpha=0) and a large weight (alpha=1.5) reduce performance relative to alpha=1.0.

Limitations

• Reliance on GPT and Claude API models for data generation, with report-only text sources that may miss imaging evidence.
• Evaluation depends on GPT-4/GPT-5 judges and an LLM-based CoT scorer, which may introduce bias.
• Dataset covers only five digestive organs from multi-institutional but partially available pathology reports.

Experimental setup

Datasets: TumorCoT-1.5M DeepTumorVQA

Benchmarks: TumorCoT-1.5M test split DeepTumorVQA

Metrics: accuracy semantic accuracy (GPT-5 judge) CoT_e (TumorChain-Eval) S_FC S_IC S_LRC

Baselines: Claude3-Haiku Gemini2.0-Flash GPT-5-Mini Qwen2.5-VL-7B InternVL-2.5-8B MiniCPM-V4.5-9B LLaVA-CoT-11B HealthGPT-3.8B Lingshu-7B MedVLM-R1-2B RadFM M3D-Phi-3-4B

Builds on: Qwen2.5-VL M3D TotalSegmentator IRCoT

Closest papers by meaning

• M3CoTBench: Benchmark Chain-of-Thought of MLLMs in Medical Image Understanding (ICLR 2026 · similarity 0.89)
• PathView-Bench: Can Multimodal Large Language Models Achieve Fine-grained Multiscale Understanding of Pathology Images? (NeurIPS 2026 · similarity 0.88)
• CXReasonBench: A Benchmark for Evaluating Structured Diagnostic Reasoning in Chest X-rays (NeurIPS 2025 · similarity 0.87)
• Gastric-X: A Multimodal Multi-Phase Benchmark Dataset for Advancing Vision-Language Models in Gastric Cancer Analysis (CVPR 2026 · similarity 0.87)
• LungNoduleAgent: A Collaborative Multi-Agent System for Precision Diagnosis of Lung Nodules (AAAI 2026 · similarity 0.87)
• Patho-R1: A Multimodal Reinforcement Learning-Based Pathology Expert Reasoner (AAAI 2026 · similarity 0.87)