Research map / Label-efficient and robust learning / Self-supervised and contrastive
XTab: Cross-table Pretraining for Tabular Transformers
Bingzhao Zhu, Xingjian Shi, Nick Erickson, Mu Li, George Karypis, Mahsa Shoaran
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Initialize FT-Transformer with cross-table pretrained weights | FT-Transformer on AMLB · AMLB downstream tasks, light finetuning | win rate: 71.0% | Improved |
| Initialize FT-Transformer with cross-table pretrained weights | FT-Transformer on AMLB · AMLB downstream tasks, heavy finetuning | win rate: 56.1% | Improved |
| Use reconstruction rather than contrastive or supervised pretraining (ablation) | XTab with FT-Transformer · AMLB downstream tasks, light and heavy finetuning | win rate: 71.0%; 56.1% | Improved |
| Use cross-table pretraining instead of random initialization | FT-Transformer on AMLB 84 tasks · AMLB 84 tasks, default hyperparameters | average rank: 10.19 ± 2.43 -> 9.21 ± 2.57 | Improved |
| Use cross-table pretraining instead of random initialization | FT-Transformer on AMLB 84 tasks · AMLB 84 tasks, default hyperparameters | average rank: 7.29 ± 2.20 -> 6.93 ± 2.09 | Improved |
| Use cross-table pretraining instead of random initialization | FT-Transformer on AMLB 84 tasks · AMLB 84 tasks, default hyperparameters | average rank: 4.94 ± 2.25 -> 4.39 ± 2.36 | Improved |
| Increase the number of cross-table pretraining tasks (ablation) | XTab with FT-Transformer · AMLB downstream tasks, light finetuning | win rate | Improved |
| Share all Transformer blocks rather than only the first block (ablation) | XTab with FT-Transformer · AMLB downstream tasks, light finetuning | win rate | Improved |
| Add the data-specific [CLS] token to the shared pretrained components (ablation) | XTab with FT-Transformer · AMLB downstream tasks, light finetuning | win rate | No clear effect |
| Reduce downstream training data to 25%, 50%, or 75% (ablation) | XTab with FT-Transformer · AMLB downstream tasks | win rate over random initialization | Improved |
| Reduce downstream training data (ablation) | XTab with FT-Transformer · AMLB downstream tasks | advantage over baseline | No clear effect |
| Use 10 local FedAvg updates per aggregation rather than fewer local updates (ablation) | XTab with FT-Transformer · AMLB downstream tasks | downstream performance | Hurt |
Closest papers by meaning
- SwitchTab: Switched Autoencoders Are Effective Tabular Learners (AAAI 2024 · similarity 0.88)
- ConTextTab: A Semantics-Aware Tabular In-Context Learner (NeurIPS 2025 · similarity 0.88)
- Large Scale Transfer Learning for Tabular Data via Language Modeling (NeurIPS 2024 · similarity 0.88)
- TabM: Advancing tabular deep learning with parameter-efficient ensembling (ICLR 2025 · similarity 0.86)
- Multimodal Tabular Reasoning with Privileged Structured Information (NeurIPS 2025 · similarity 0.86)
- CARTE: Pretraining and Transfer for Tabular Learning (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
