Research map / Vision-language and multimodal / CLIP and zero-shot
Disease-Centric Vision-Language Pretraining with Hybrid Visual Encoding for 3D Computed Tomography
Bowen Shi, Weiwei Cao, Ruifeng Yuan, Wanxing Chang, Wenrui Dai, Hongkai Xiong, Ling Zhang, Jianpeng Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace plain CNN encoder with CNN-ViT hybrid encoder (ablation) | CT-DiagVLM base model · CT-RATE | AUC: 79.3 -> 80.3 | Improved |
| add hybrid encoder when disease loss and prompt are already enabled (ablation) | CT-DiagVLM · CT-RATE | AUC: 83.2 -> 84.4 | Improved |
| add diagnosis-aware prompt strategy on top of hybrid encoder (ablation) | CT-DiagVLM · CT-RATE | AUC: 80.3 -> 82.4 | Improved |
| use handcrafted prompt refinement instead of diagnosis-aware prompt (ablation) | CT-DiagVLM · CT-RATE | AUC: 80.3 -> 81.0 | Improved |
| add disease-level contrastive learning (ablation) | CT-DiagVLM · CT-RATE | AUC: 82.4 -> 84.4 | Improved |
| use per-disease contrastive loss instead of coarse disease co-occurrence matching (ablation) | CT-DiagVLM · CT-RATE | AUC: 83.2 -> 84.4 | Improved |
| add DeepStack to hybrid encoder (ablation) | CT-DiagVLM · CT-RATE | AUC: 84.4 -> 84.0 | Hurt |
| add organ-level MoE to hybrid encoder (ablation) | CT-DiagVLM · CT-RATE | AUC: 84.4 -> 84.4 | No clear effect |
| initialize ViT from pre-trained SigLIP2 instead of from scratch (ablation) | CT-DiagVLM hybrid encoder · CT-RATE | AUC: 76.5 -> 84.4 | Improved |
| use larger SigLIP2 So-400M or medical MedSigLIP instead of SigLIP2 Base (ablation) | CT-DiagVLM hybrid encoder · CT-RATE | AUC: 84.4 -> 84.2 (So-400M), 83.8 (MedSigLIP) | No clear effect |
| increase number of retrieved reports M for disease prototypes to 500 or more (ablation) | diagnosis-aware prompt · CT-RATE | AUC: plateau at M>=500, peak 84.4 | Improved |
| CT-DiagVLM versus base model on 60-disease setting | CT-DiagVLM · CT-RATE 60 diseases | sensitivity: 72.0 -> 65.5 | Hurt |
Closest papers by meaning
- Large-scale and Fine-grained Vision-language Pre-training for Enhanced CT Image Understanding (ICLR 2025 · similarity 0.92)
- Better Tokens for Better 3D: Advancing Vision-Language Modeling in 3D Medical Imaging (NeurIPS 2025 · similarity 0.91)
- Toward a Vision-Language Foundation Model for Medical Data: Multimodal Dataset and Benchmarks for Vietnamese PET/CT Report Generation (NeurIPS 2025 · similarity 0.89)
- 3D-RAD: A Comprehensive 3D Radiology Med-VQA Dataset with Multi-Temporal Analysis and Diverse Diagnostic Tasks (NeurIPS 2025 · similarity 0.88)
- Bootstrapping Chest CT Image Understanding by Distilling Knowledge from X-ray Expert Models (CVPR 2024 · similarity 0.88)
- Enhancing vision-language models for medical imaging: bridging the 3D gap with innovative slice selection (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
