Research map / Vision-language and multimodal / Cross-modal learning
Towards Unifying Medical Vision-and-Language Pre-Training via Soft Prompts
Zhihong Chen, Shizhe Diao, Benyou Wang, Guanbin Li, Xiang Wan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add ITC objective to MLM+ITM pre-training (ablation) | PTUnifier · CheXpert 1% data | AUROC: 74.5 -> 88.7 | Improved |
| add ITC objective to MLM+ITM pre-training (ablation) | PTUnifier · RadNLI text classification | accuracy: 78.3 -> 80.0 | Improved |
| use ITC only instead of MLM+ITM+ITC for text classification (ablation) | PTUnifier · RadNLI text classification | accuracy: 78.3 -> 76.5 | Hurt |
| add MLM and ITM fusion objectives to ITC (ablation) | PTUnifier · VQA-RAD overall | accuracy: 74.6 -> 78.3 | Improved |
| add ITM to MLM (ablation) | PTUnifier · VQA-RAD overall | accuracy: 70.8 -> 77.7 | Improved |
| add MLM and ITM to ITC on image-to-text generation (ablation) | PTUnifier · MIMIC-CXR image-to-text generation | BLEU-4: 10.3 -> 10.7 | Improved |
| unify Med-VLP with visual and textual prompt pools versus task-specific state-of-the-art | PTUnifier · ROCO image-to-text retrieval | Recall@1: 14.5 -> 21.0 | Improved |
| unify Med-VLP with prompt pools versus task-specific state-of-the-art on image generation | PTUnifier · MIMIC-CXR image-to-text generation | BLEU-4: 10.5 -> 10.7 | Improved |
| unify Med-VLP with prompt pools versus task-specific state-of-the-art on VQA | PTUnifier · VQA-RAD | accuracy: 72.7 -> 78.3 | Improved |
Closest papers by meaning
- MedUnifier: Unifying Vision-and-Language Pre-training on Medical Data with Vision Generation Task using Discrete Visual Representations (CVPR 2025 · similarity 0.93)
- MIMO: A Medical Vision Language Model with Visual Referring Multimodal Input and Pixel Grounding Multimodal Output (CVPR 2025 · similarity 0.89)
- Alignment, Mining and Fusion: Representation Alignment with Hard Negative Mining and Selective Knowledge Fusion for Medical Visual Question Answering (CVPR 2025 · similarity 0.89)
- Fine-grained Adaptive Visual Prompt for Generative Medical Visual Question Answering (AAAI 2025 · similarity 0.89)
- BiomedCoOp: Learning to Prompt for Biomedical Vision-Language Models (CVPR 2025 · similarity 0.88)
- Uni-Med: A Unified Medical Generalist Foundation Model For Multi-Task Learning Via Connector-MoE (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
