Research map / Vision-language and multimodal / Cross-modal learning
TIP: Tabular-Image Pre-training for Multimodal Classification with Incomplete Data
Siyi Du, Shaoming Zheng, Yinsong Wang, Wenjia Bai, Declan P. O'Regan, Chen Qin
This paper tried a common change: Add a contrastive loss
add image-tabular contrastive loss (ITC) to pre-training · Infarction linear probing · AUC (%): 70.38 -> 84.46 Improved
What 1,317 accepted papers report for this change
- 1,253 improved
- 22 no clear effect
- 16 mixed
- 26 hurt
- Beyond Looking Up, Try Looking Around: Harmonizing Global Structure and Local Consistency in Optimal Transport for Short Text Clustering ICML 2026 · add contrastive learning · SearchSnippets · ACC: 56.65 -> 80.12Improved
- SleepLM: Natural-Language Intelligence for Human Sleep ICML 2026 · add contrastive objective to captioning (CoCa vs Cap) · sleep event IoU · IoU: 29.2 -> 27.4Hurt
- LIMSSR: LLM-Driven Sequence-to-Score Reasoning under Training-Time Incomplete Multimodal Observations ICML 2026 · add contrastive loss L_con · FS1000 average · Spearman correlation: 0.759 -> 0.789Improved
See this change in detailMatched by the wording of each paper's own result, read by AI from the full text. Papers report gains more often than failures, so this shows what gets published, not the chance a change works.
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SSL image-tabular pre-training (ITC+ITM+MTR) (ablation) | TIP architecture · DVM linear probing | accuracy (%): 98.57 -> 99.72 | Improved |
| add learnable column name embeddings to tabular encoder (ablation) | TIP · CAD linear probing | AUC (%): 79.40 -> 86.43 | Improved |
| add classifier ensemble during fine-tuning (ablation) | TIP · Infarction fine-tuning | AUC (%): 84.00 -> 85.58 | Improved |
| add classifier ensemble during fine-tuning (ablation) | TIP · CAD fine-tuning | AUC (%): 86.97 -> 86.03 | Mixed |
| add image-tabular contrastive loss (ITC) to pre-training (ablation) | TIP · Infarction linear probing | AUC (%): 70.38 -> 84.46 | Improved |
| use larger ViT-B instead of ViT-S image encoder (ablation) | TIP · DVM linear probing | accuracy (%): 99.67 -> 99.40 | No clear effect |
| use masking ratio 0.5-0.7 instead of 0.1 or 0.9 in masked tabular reconstruction (ablation) | TIP · DVM RMSE, missing rate 0.5 | RMSE: 0.6752 (rho 0.1) -> 0.4651 (rho 0.5) | Improved |
| multimodal pre-training with tabular branch in fine-tuning instead of MMCL image-only downstream use | MMCL · CAD linear probing | AUC (%): 74.71 -> 86.43 | Improved |
| compare TIP to supervised multimodal methods on CAD and Infarction | TIP · Infarction linear probing | AUC (%): 85.05 (Concat Fuse) vs 84.46 | No clear effect |
| use TIP vs imputation methods for continuous feature reconstruction | TIP · DVM RMSE sigma 0.5 | RMSE: 0.7653 -> 0.4651 | Improved |
Closest papers by meaning
- STiL: Semi-supervised Tabular-Image Learning for Comprehensive Task-Relevant Information Exploration in Multimodal Classification (CVPR 2025 · similarity 0.90)
- MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image (NeurIPS 2026 · similarity 0.86)
- Semi-Supervised Multimodal Classification Through Learning from Modal and Strategic Complementarities (AAAI 2025 · similarity 0.85)
- Tabular Insights, Visual Impacts: Transferring Expertise from Tables to Images (ICML 2024 · similarity 0.85)
- Asymmetric Reinforcing Against Multi-Modal Representation Bias (AAAI 2025 · similarity 0.85)
- Large Scale Transfer Learning for Tabular Data via Language Modeling (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
