Research map / Label-efficient and robust learning / Recognition and long-tail
ICE-T: Interactions-aware Cross-column Contrastive Embedding for Heterogeneous Tabular Datasets
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ICE-T, a contrastive embedding method for heterogeneous tabular data that treats each column as a modality and contrasts each column embedding against the mean embedding of the other columns, scaling linearly in the number of columns.
Problem
Learning high-quality representations of heterogeneous tabular datasets with contrastive learning; existing pairwise multimodal contrastive losses scale quadratically with the number of columns and fail to capture interactions among variables.
Approach
Treat each tabular column as a modality, compute per-column intermediate embeddings, and contrast each column's embedding against the arithmetic mean of the other columns' intermediate embeddings (leave-one-out anchor), giving linear scaling and capturing cross-column interactions while allowing column-specific and partial-instance embeddings.
Key results
- ICE-T achieves average relative score 0.824 in transfer learning versus 0.544 for the best baseline (GMC and CLIP).
- On the noisy XOR synthetic task, ICE-T imputation score 0.982 versus 0.589 for the best baseline (MCN).
- ICE-T has the best imputation (0.604) and supervised learning (0.687) average relative scores, while Scarf leads clustering (0.617 for ICE-T is second).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ICE-T instead of the strongest reported baseline | Gaussian XOR blobs · Gaussian XOR blobs | Imputation score: 0.589 -> 0.982 | Helped |
| Use ICE-T instead of CLIP | Gaussian XOR blobs · Gaussian XOR blobs | ACC (c): 0.750 -> 0.900 | Helped |
| Use ICE-T instead of GMC | Gaussian XOR blobs · Gaussian XOR blobs | MSE (x): 0.206 -> 0.008 | Helped |
| Use ICE-T instead of CLIP | Gaussian XOR blobs · Gaussian XOR blobs | MSE (y): 0.106 -> 0.006 | Helped |
| Use ICE-T instead of CLIP | tabular contrastive representation learning · 48 real-world datasets | Imputation: 0.561 -> 0.604 | Helped |
| Use ICE-T instead of GMC | tabular contrastive representation learning · 48 real-world datasets | Clustering: 0.553 -> 0.617 | Helped |
| Use ICE-T instead of MCN | tabular contrastive representation learning · 48 real-world datasets | Supervised learning: 0.636 -> 0.687 | Helped |
| Use ICE-T instead of CLIP | tabular contrastive representation learning · 48 real-world datasets | Transfer learning: 0.544 -> 0.824 | Helped |
Surprising findings
- Pairwise multimodal contrastive methods (CLIP, GMC, MCN) fail to capture variable interactions on the noisy XOR dataset, with imputation accuracy near chance for some variables.
- Scarf beats ICE-T in clustering, which the authors attribute to instance-level global alignment.
Limitations
- CLIP and MCN were excluded from datasets with more than 25 variables due to computational cost.
- Clustering performance is weaker than Scarf.
- One large dataset (delays zurich airport) was excluded due to computational restrictions.
Experimental setup
Datasets: Grinsztajn et al. tabular benchmark (44 datasets) image-tabular datasets (skin cancer, streetview) text-tabular datasets (clothing, kickstarter) synthetic noisy XOR blobs
Benchmarks: 48 real-world tabular datasets (imputation, clustering, supervised KNN, transfer learning) noisy XOR synthetic dataset
Metrics: balanced accuracy MSE silhouette score average relative score
Baselines: Scarf SubTab CLIP GMC MCN XGBoost k-means on raw data KNN on raw data
Builds on: CLIP GMC MCN Scarf SubTab InfoNCE
Closest papers by meaning
- TRL-Bench: Standardizing Cross-Paradigm Representation-Level Evaluation of Tabular Encoders (NeurIPS 2026 · similarity 0.88)
- Enhancing Tabular Learners with Context-Aware Semantic Embeddings (NeurIPS 2026 · similarity 0.87)
- SwitchTab: Switched Autoencoders Are Effective Tabular Learners (AAAI 2024 · similarity 0.87)
- ConTextTab: A Semantics-Aware Tabular In-Context Learner (NeurIPS 2025 · similarity 0.87)
- Large Scale Transfer Learning for Tabular Data via Language Modeling (NeurIPS 2024 · similarity 0.87)
- MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.