Research map / Vision-language and multimodal / Cross-modal learning

MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image

NeurIPS 2026 · Vision-language and multimodal › Cross-modal learning · approach: Unimodal-to-multimodal representation transfer · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces MulTaBench, a 40-dataset benchmark (20 image-tabular, 20 text-tabular) curated with a pipeline requiring joint signal and target-aware representation gains over frozen embeddings, and releases 40 extra candidate datasets.

Problem

Existing multimodal tabular learning benchmarks mostly test mere co-occurrence of modalities, so they mix tasks where frozen embeddings suffice with tasks needing task-specific (target-aware) representations; image-tabular benchmarks are also scarce.

Approach

A curation pipeline that keeps datasets only if (1) joint frozen embeddings beat both unimodal variants (joint signal) and (2) LoRA-finetuning the encoder on the target (target-aware representations) improves over frozen embeddings across tabular learners, yielding a benchmark where target-aware adaptation matters.

Key results

• Target-aware representations outperform frozen embeddings for all 5 curation learners on text-tabular candidates, and the curated MulTaBench shows significant TAR gains on 37 of 40 datasets.
• Of text-tabular candidates, about 23% fail joint signal and 36% of the remainder fail task-awareness, leaving 41% passing both.
• Only 5 of 16 image-tabular literature candidates (31%) passed the pipeline, leading to 20 curated image-tabular datasets, the largest image-tabular benchmark to date.

Surprising findings

• TAR with e5/DINO-small embeddings outperforms frozen Large embeddings, indicating that tuning matters more than embedding capacity.
• ConTextTab, state of the art on the CARTE benchmark, has the worst performance among text-tabular methods on MulTaBench.

Limitations

• Curation pipeline entangles the computational problem with the algorithmic solution, making dataset suitability hard to predict in advance.
• Selection bias: curation learners cannot be fairly evaluated on the benchmark.
• Hyperparameters fixed without per-dataset tuning; finetuning adds high computational overhead.

Experimental setup

Datasets: 40 MulTaBench image-tabular and text-tabular datasets 56 text-tabular benchmark datasets (curation pool) 16 image-tabular candidate datasets from literature Kaggle image-tabular datasets

Benchmarks: MulTaBench

Metrics: AUC R^2 min-max normalized score

Baselines: LightGBM CatBoost TabM TabPFNv2 TabPFN-2.5 XGBoost RandomForest RealMLP TabDPT TabICLv2 TabSTAR ConTextTab

Builds on: TabPFN TabSTAR ConTextTab AutoGluon-Multimodal TabM

Closest papers by meaning

• VT-Bench: A Unified Benchmark for Visual-Tabular Multi-Modal Learning (ICML 2026 · similarity 0.90)
• TRL-Bench: Standardizing Cross-Paradigm Representation-Level Evaluation of Tabular Encoders (NeurIPS 2026 · similarity 0.89)
• TabDPT: Scaling Tabular Foundation Models on Real Data (NeurIPS 2025 · similarity 0.89)
• Tabular Foundation Model for Generative Modelling (NeurIPS 2026 · similarity 0.88)
• Multimodal Tabular Reasoning with Privileged Structured Information (NeurIPS 2025 · similarity 0.88)
• TabSTAR: A Tabular Foundation Model for Tabular Data with Text Fields (NeurIPS 2025 · similarity 0.88)