Research map / Vision-language and multimodal / Multimodal LLMs

VT-Bench: A Unified Benchmark for Visual-Tabular Multi-Modal Learning

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces VT-Bench, a unified benchmark of 14 vision-tabular datasets (756K+ samples, 9 domains) spanning discriminative prediction and generative reasoning, evaluating 23 models and proposing MCR/MIR modality diagnostics.

Problem

Vision-tabular multi-modal learning is underexplored, and existing benchmarks are narrow in domain and either render tables as images or lack structure-aware and numerical reasoning evaluation.

Approach

A unified benchmark covering both discriminative prediction (classification/regression with image+tabular row) and generative reasoning (table/database QA with images), plus two modality diagnostics, Modality Contribution Ratio (MCR) and Modality Informativeness Ratio (MIR), to analyze fusion and negative transfer.

Key results

• Proprietary Gemini-3-Flash-Preview reaches about 72% average accuracy on DVM-Car QA, 70% on MMQA, and 27% on FULL EHRXQA.
• Multi-modal vision-tabular models do not consistently beat the strongest unimodal baselines; negative transfer appears across many model-dataset pairs (e.g., TIP, DAFT, Concat).
• Open-source models show a drop from Identification to Row Localization to Attribute Retrieval on DVM-Car QA, and all models struggle on Constrained Counting and Conditional Mean.

Surprising findings

• Removing either modality often improves performance for multi-modal fusion models, i.e., negative transfer is prevalent.
• Vision-language pretrained models underperform on vision-tabular tasks, and contrastive alignment (MMCL) yields inconsistent gains.

Limitations

• Conclusions restricted to the evaluated datasets, tasks, and models; medical-centric domain coverage.
• Does not cover other tabular formats (CSV, HTML, SQL, Markdown, table images) in separate splits.
• No theoretical analysis linking MCR/MIR to model performance.

Experimental setup

Datasets: Breast Cancer Skin Cancer Infarction Pneumonia Adoption CelebA DVM-Car Los Respiratory Rate Pawpularity Anime EHRXQA

Benchmarks: VT-Bench EHRXQA MMQA DVM-Car QA

Metrics: Accuracy AUC Macro-F1 RMSE MAE R^2 exact-match accuracy MCR MIR

Baselines: ResNet-50 ViT-B/16 LightGBM TabTransformer TabPFN-v2 Concat MAX MUL DAFT CHARMS MMCL TIP

Builds on: DAFT CHARMS TIP MMCL StructGPT Thyme EHRXQA Multi-ModalQA

Closest papers by meaning

• MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image (NeurIPS 2026 · similarity 0.90)
• TABLET: A Large-Scale Dataset for Robust Visual Table Understanding (ICLR 2026 · similarity 0.88)
• TRL-Bench: Standardizing Cross-Paradigm Representation-Level Evaluation of Tabular Encoders (NeurIPS 2026 · similarity 0.88)
• TabArena: A Living Benchmark for Machine Learning on Tabular Data (NeurIPS 2025 · similarity 0.87)
• TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergy (NeurIPS 2024 · similarity 0.87)
• Multimodal Tabular Reasoning with Privileged Structured Information (NeurIPS 2025 · similarity 0.87)