Research map / Label-efficient and robust learning / Self-supervised and contrastive
Best of Both Worlds: Multimodal Contrastive Learning With Tabular and Imaging Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Pretrain image encoders jointly with paired tabular data using cross-modal contrastive learning | ResNet50 image encoder · Myocardial infarction | AUC (%) / frozen: 72.37$\pm$1.80 -> 76.35$\pm$0.19 | Helped |
| Pretrain image encoders jointly with paired tabular data using cross-modal contrastive learning | ResNet50 image encoder · CAD | AUC (%) / frozen: 69.86$\pm$0.21 -> 74.45$\pm$0.09 | Helped |
| Pretrain image encoders jointly with paired tabular data using cross-modal contrastive learning | ResNet50 image encoder · DVM car model | Top-1 accuracy (%) / frozen: 87.97$\pm$2.20 -> 91.43$\pm$0.13 | Helped |
| Append the target label as a tabular feature during multimodal pretraining | Multimodal Imaging · DVM car model | Top-1 accuracy (%) / frozen: 91.43$\pm$0.13 -> 93.56$\pm$0.08 | Helped |
| Combine label-as-a-feature with false negative elimination | Multimodal Imaging · DVM car model | Top-1 accuracy (%) / frozen: 92.39$\pm$0.18 -> 94.07$\pm$0.05 | Helped |
| Combine label-as-a-feature with supervised contrastive learning | Multimodal Imaging · DVM car model | Top-1 accuracy (%) / frozen: 93.82$\pm$0.11 -> 94.40$\pm$0.04 | Helped |
| Append the target label as a tabular feature during multimodal pretraining | Multimodal Imaging · Myocardial infarction | AUC (%) / frozen: 76.35$\pm$0.19 -> 76.60$\pm$0.42 | No clear effect |
| Use only morphometric tabular features (ablation) | Multimodal Imaging · DVM car model | Top-1 accuracy (%) / frozen: 91.43$\pm$0.13 -> 92.33$\pm$0.05 | Helped |
| Add morphometric tabular features rather than excluding them (ablation) | Multimodal Imaging · DVM car model | Top-1 accuracy (%) / frozen: 89.14$\pm$0.24 -> 91.43$\pm$0.13 | Helped |
| Use only morphometric tabular features (ablation) | Multimodal Imaging · Myocardial infarction | AUC (%) / frozen: 76.35$\pm$0.19 -> 75.22$\pm$0.30 | Hurt |
| Add morphometric tabular features rather than excluding them (ablation) | Multimodal Imaging · CAD | AUC (%) / frozen: 72.18$\pm$0.25 -> 74.45$\pm$0.09 | Helped |
| Use only morphometric tabular features (ablation) | Multimodal Imaging · CAD | AUC (%) / frozen: 74.45$\pm$0.09 -> 73.71$\pm$0.09 | Hurt |
Closest papers by meaning
- MedTrinity-25M: A Large-scale Multimodal Dataset with Multigranular Annotations for Medicine (ICLR 2025 · similarity 0.85)
- Large-scale and Fine-grained Vision-language Pre-training for Enhanced CT Image Understanding (ICLR 2025 · similarity 0.85)
- MLIP: Enhancing Medical Visual Representation with Divergence Encoder and Knowledge-guided Contrastive Learning (CVPR 2024 · similarity 0.85)
- A Learnable Discrete-Prior Fusion Autoencoder with Contrastive Learning for Tabular Data Synthesis (AAAI 2024 · similarity 0.85)
- Contrasting with Symile: Simple Model-Agnostic Representation Learning for Unlimited Modalities (NeurIPS 2024 · similarity 0.84)
- Multi-modal Vision Pre-training for Medical Image Analysis (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.