Research map / Label-efficient and robust learning / Recognition and long-tail
Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Models
Xiyuan Zhang, Danielle C. Maddix, Junming Yin, Nick Erickson, Abdul Fatir Ansari, Boran Han, Shuai Zhang, Leman Akoglu, Christos Faloutsos, Michael W. Mahoney, Cuixiong Hu, Huzefa Rangwala and 2 more
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Systematically analyzes synthetic data priors for tabular foundation models and builds Mitra, a 2D-attention TFM pretrained on a mixture of SCM and tree-based priors that outperforms TabPFNv2 and TabICL on classification and regression benchmarks.
Problem
Tabular foundation models based on in-context learning are pretrained on synthetic data, but the principles for designing the synthetic priors (what makes a good prior and how to mix them) remain poorly understood.
Approach
Identify three prior-quality factors (standalone performance on real data, diversity, and distinctiveness within a mixture) via a generalizability matrix, then build a mixture of SCM and tree-based priors (DT, ET, GB, RF, plus a directly sampled RF) to pretrain a TFM that is model-agnostic across 1D and 2D attention.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the full mixed-prior method with ensembling and fine-tuning | Mitra · Merged classification benchmarks | Elo: 1015 -> 1136 | Improved |
| Use the full mixed-prior method with ensembling and fine-tuning | Mitra · TabRepo regression | Elo: 963 -> 1140 | Improved |
| Add ET prior to SCM (ablation) | Mitra prior mixture · TabRepo classification | Elo: 1000 -> 1063 (+63) | Improved |
| Add GB prior to SCM + ET (ablation) | Mitra prior mixture · TabRepo classification | Elo: 1063 -> 1076 | Improved |
| Add RF prior to SCM + ET + GB (ablation) | Mitra prior mixture · TabRepo classification | Elo: 1076 -> 1050 | No clear effect |
| Add DSRF prior to SCM + ET + GB (ablation) | Mitra prior mixture · TabRepo classification | Elo: 1076 -> 1046 | No clear effect |
| Add DT prior to SCM + ET + GB (ablation) | Mitra prior mixture · TabRepo classification | Elo: 1076 -> 1059 | No clear effect |
| Add DT prior to SCM + ET (ablation) | Mitra prior mixture · TabRepo classification | Elo: 1063 -> 1028 | Hurt |
| Add RF prior to SCM (ablation) | Mitra prior mixture · TabRepo classification | Elo: 1000 -> 1005 | No clear effect |
| Add DT prior to SCM (ablation) | Mitra prior mixture · TabRepo classification | Elo: 1000 -> 1025 | Improved |
| Add GB prior to SCM (ablation) | Mitra prior mixture · TabRepo classification | Elo: 1000 -> 1028 | Improved |
| Add DSRF prior to SCM (ablation) | Mitra prior mixture · TabRepo classification | Elo: 1000 -> 1058 | Improved |
Experimental setup
Baselines: TabPFNv2 TabICL Attic TabPFN TabForestPFN RealMLP AutoGluon LightGBM XGBoost CatBoost
Builds on: TabPFN TabPFNv2 Attic TabForestPFN
Closest papers by meaning
- TabDPT: Scaling Tabular Foundation Models on Real Data (NeurIPS 2025 · similarity 0.90)
- Understanding the Surprising Generalization Properties of Tabular Foundation Models (NeurIPS 2026 · similarity 0.89)
- Tabular Foundation Model for Generative Modelling (NeurIPS 2026 · similarity 0.89)
- TabICL: A Tabular Foundation Model for In-Context Learning on Large Data (ICML 2025 · similarity 0.87)
- TabArena: A Living Benchmark for Machine Learning on Tabular Data (NeurIPS 2025 · similarity 0.87)
- TabPrep: Closing the Feature Engineering Gap in Tabular Benchmarks (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
