Research map / Vision-language and multimodal / Human and embodied modeling
MedVTok: A General-Purpose Medical Visual Tokenizer
Chenglong Ma, Yuanfeng Ji, Junzhi Ning, Jiyao Liu, Yujie Wei, Shaohao Rui, Chaoyang Zhang, Wenjie Li, Wenhao Tang, Cheng Tang, Wei Li, Shujian Gao and 9 more
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MedVTok, a general-purpose visual tokenizer intended to map 2D medical images and 3D volumes into a shared token interface for multimodal medical AI.
Problem
Existing medical visual tokenizers are tied to a specific dimensionality (2D or 3D) and imaging modality, so multimodal medical systems rely on fragmented, poorly reusable representations.
Approach
A single visual tokenizer that unifies 2D images and 3D volumes across imaging modalities into a common visual interface for patient condition modeling. Only the abstract was available, so the specific mechanism is not stated.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multi-expert shared-space alignment instead of a single expert (MedImageInsight) (ablation) | MedVTok (stage 1, 3M images) · classification and segmentation | accuracy / Dice | Improved |
| add shared-space contrastive alignment with a single expert (ablation) | MedVTok (stage 1, 3M images) · classification and segmentation | accuracy / Dice | Improved |
| add expert-space complementary alignment L_expert (ablation) | MedVTok (stage 1, 3M images) · classification | accuracy: +1.53% | Improved |
| replace attention pooling with mean pooling plus linear layer (ablation) | MedVTok aggregation head · segmentation and classification | Dice / accuracy | Improved |
| scale training data from 3M to 28M images (ablation) | MedVTok · downstream tasks | classification and segmentation | Improved |
| add slice differential regularization (SliDR) (ablation) | MedVTok stage 2 · 3D reconstruction | PSNR / SSIM | Improved |
| align to multiple experts instead of text-supervised semantic encoder (no image-text pairs) | MedVTok vs MedITok · VQA average (LLaVA-Med) | accuracy (%): 45.39 -> 46.48 | Improved |
| use MedVTok tokens instead of MedITok tokens | LaMed report generation · AMOS-MM GREEN avg | GREEN: 0.386 -> 0.413 | Improved |
| MedVTok vs specialist expert on pathology | MedVTok · Pathology classification and segmentation | accuracy / Dice: Dice 77.74 vs Conch 83.77 | Hurt |
| MedVTok vs MedSigLIP on pathology and X-ray segmentation | MedVTok · Linear segmentation | Dice: X-ray 61.13 vs 62.48 | Mixed |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- UniTok — UniTok: a Unified Tokenizer for Visual Generation and Understanding (NeurIPS 2025) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Better Tokens for Better 3D: Advancing Vision-Language Modeling in 3D Medical Imaging (NeurIPS 2025 · similarity 0.88)
- MedMO: Grounding and Understanding Multimodal Large Language Model for Medical Images (CVPR 2026 · similarity 0.88)
- Unlocking the Power of Spatial and Temporal Information in Medical Multimodal Pre-training (ICML 2024 · similarity 0.88)
- Interpretable Bilingual Multimodal Large Language Model for Diverse Biomedical Tasks (ICLR 2025 · similarity 0.87)
- MedAtlas: Evaluating LLMs for Multi-Round, Multi-Task Medical Reasoning Across Diverse Imaging Modalities and Clinical Text (AAAI 2026 · similarity 0.87)
- MedTrinity-25M: A Large-scale Multimodal Dataset with Multigranular Annotations for Medicine (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
