AI Paper Atlas

Research map / Vision-language and multimodal / Cross-modal learning

TIP: Tabular-Image Pre-training for Multimodal Classification with Incomplete Data

Siyi Du, Shaoming Zheng, Yinsong Wang, Wenjia Bai, Declan P. O'Regan, Chen Qin

ECCV 2024 · Vision-language and multimodal › Cross-modal learning · approach: Multimodal fusion with missing modalities · 24 citations (OpenAlex)

This paper tried a common change: Add a contrastive loss

add image-tabular contrastive loss (ITC) to pre-training · Infarction linear probing · AUC (%): 70.38 -> 84.46 Improved

What 1,317 accepted papers report for this change

See this change in detailMatched by the wording of each paper's own result, read by AI from the full text. Papers report gains more often than failures, so this shows what gets published, not the chance a change works.

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add SSL image-tabular pre-training (ITC+ITM+MTR) (ablation)TIP architecture · DVM linear probingaccuracy (%): 98.57 -> 99.72Improved
add learnable column name embeddings to tabular encoder (ablation)TIP · CAD linear probingAUC (%): 79.40 -> 86.43Improved
add classifier ensemble during fine-tuning (ablation)TIP · Infarction fine-tuningAUC (%): 84.00 -> 85.58Improved
add classifier ensemble during fine-tuning (ablation)TIP · CAD fine-tuningAUC (%): 86.97 -> 86.03Mixed
add image-tabular contrastive loss (ITC) to pre-training (ablation)TIP · Infarction linear probingAUC (%): 70.38 -> 84.46Improved
use larger ViT-B instead of ViT-S image encoder (ablation)TIP · DVM linear probingaccuracy (%): 99.67 -> 99.40No clear effect
use masking ratio 0.5-0.7 instead of 0.1 or 0.9 in masked tabular reconstruction (ablation)TIP · DVM RMSE, missing rate 0.5RMSE: 0.6752 (rho 0.1) -> 0.4651 (rho 0.5)Improved
multimodal pre-training with tabular branch in fine-tuning instead of MMCL image-only downstream useMMCL · CAD linear probingAUC (%): 74.71 -> 86.43Improved
compare TIP to supervised multimodal methods on CAD and InfarctionTIP · Infarction linear probingAUC (%): 85.05 (Concat Fuse) vs 84.46No clear effect
use TIP vs imputation methods for continuous feature reconstructionTIP · DVM RMSE sigma 0.5RMSE: 0.7653 -> 0.4651Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy