Research map / Architectures and efficiency / Attention and transformers
MultiTab: A Scalable Foundation for Multitask Learning on Tabular Data
Dimitrios Sinodinos, Jack Yi Wei, Narges Armanfard
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add T-/->T masking (task tokens cannot attend to each other) to multi-token model (ablation) | MultiTab-Net · Higgs | multitask gain Δm: 1.1182 -> 1.2337 | Improved |
| add T-/->T masking to multi-token model (ablation) | MultiTab-Net · AliExpress | multitask gain Δm: 0.2579 -> 0.5512 | Improved |
| add T-/->T masking to multi-token model (ablation) | MultiTab-Net · ACS Income | multitask gain Δm: 0.0783 -> 0.1064 | Improved |
| use multiple task tokens instead of one shared token (no masking) (ablation) | MultiTab-Net · Higgs | multitask gain Δm: -6.3491 -> 1.1182 | Improved |
| use multiple task tokens instead of one shared token (no masking) (ablation) | MultiTab-Net · AliExpress | multitask gain Δm: 0.2669 -> 0.2579 | No clear effect |
| use multiple task tokens instead of one shared token (no masking) (ablation) | MultiTab-Net · ACS Income | multitask gain Δm: 0.0893 -> 0.0783 | No clear effect |
| add F-/->T masking to multi-token model (ablation) | MultiTab-Net · Higgs | multitask gain Δm: 1.1182 -> 0.9626 | Hurt |
| add F-/->T masking to multi-token model (ablation) | MultiTab-Net · AliExpress | multitask gain Δm: 0.2579 -> 0.3698 | Improved |
| add F-/->T masking to single-token model (ablation) | MultiTab-Net · Higgs | multitask gain Δm: -6.3491 -> -12.5587 | Hurt |
| add F-/->T masking to single-token model (ablation) | MultiTab-Net · AliExpress | multitask gain Δm: 0.2669 -> 0.1301 | Hurt |
| add F-/->T & T-/->T masking to multi-token model (ablation) | MultiTab-Net · Higgs | multitask gain Δm: 1.1182 -> 1.0197 | Hurt |
| use MultiTab-Net instead of STEM | MultiTab-Net vs STEM · Higgs | multitask gain Δm: 0.0571 -> 1.2337 | Improved |
Closest papers by meaning
- TabDPT: Scaling Tabular Foundation Models on Real Data (NeurIPS 2025 · similarity 0.87)
- TabFlex: Scaling Tabular Learning to Millions with Linear Attention (ICML 2025 · similarity 0.87)
- ConTextTab: A Semantics-Aware Tabular In-Context Learner (NeurIPS 2025 · similarity 0.86)
- TabICL: A Tabular Foundation Model for In-Context Learning on Large Data (ICML 2025 · similarity 0.86)
- Large Scale Transfer Learning for Tabular Data via Language Modeling (NeurIPS 2024 · similarity 0.86)
- Understanding the Surprising Generalization Properties of Tabular Foundation Models (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
