Research map / Label-efficient and robust learning / Recognition and long-tail
PTaRL: Prototype-based Tabular Representation Learning via Space Calibration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PTaRL, a model-agnostic prototype-based projection space that projects deep tabular model representations onto K-Means-initialized global prototypes via optimal transport, with diversification and orthogonality constraints, improving six deep tabular models on six benchmarks.
Problem
Deep tabular models suffer from representation entanglement and localization (samples represented independently, ignoring global data structure), leading to inconsistent performance across tasks such as classification and regression.
Approach
Build a prototype-based projection space whose basis vectors are global prototypes (K-Means-initialized), project each sample's backbone representation onto it by minimizing an optimal transport distance, and further disentangle representations with a coordinate diversification (contrastive-style) loss and a prototype matrix orthogonalization loss.
Key results
- FT-Transformer + PTaRL reaches 0.871 accuracy on Adult vs 0.827 for FT-Transformer alone, and 0.738 vs 0.687 on Higgs.
- PTaRL improves all six deep tabular backbones on all six datasets, with gains reported as more than 4% in every setting and Wilcoxon signed-rank significance at alpha=0.05.
- On California housing, FT-Transformer RMSE drops from 0.486 to 0.448 with PTaRL; ablation shows removing orthogonalization and diversification each degrades results.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add PTaRL to FT-Transformer | FT-Transformer · Adult, Higgs, Helena, Jannis, ALOI, California housing | accuracy / RMSE: Adult: 0.827 -> 0.871; Higgs: 0.687 -> 0.738; Helena: 0.352 -> 0.397; Jannis: 0.689 -> 0.738; ALOI: 0.924 -> 0.97; Calif | Helped |
| Add prototype matrix orthogonalization (ablation) | PTaRL with FT-Transformer · Adult, Higgs, Helena, Jannis, ALOI, California housing | accuracy / RMSE: Adult: 0.859 -> 0.871; Higgs: 0.722 -> 0.738; Helena: 0.383 -> 0.397; Jannis: 0.725 -> 0.738; ALOI: 0.96 -> 0.97; Califo | Helped |
| Add coordinates diversifying alongside prototype matrix orthogonalization (ablation) | PTaRL with FT-Transformer · Adult, Higgs, Helena, Jannis, ALOI, California housing | accuracy / RMSE: Adult: 0.841 -> 0.871; Higgs: 0.704 -> 0.738; Helena: 0.369 -> 0.397; Jannis: 0.702 -> 0.738; ALOI: 0.94 -> 0.97; Califo | Helped |
| Add PTaRL's prototype-based projection to the backbone (ablation) | FT-Transformer · Adult, Higgs, Helena, Jannis, ALOI, California housing | accuracy / RMSE: Adult: 0.827 -> 0.871; Higgs: 0.687 -> 0.738; Helena: 0.352 -> 0.397; Jannis: 0.689 -> 0.738; ALOI: 0.924 -> 0.97; Calif | Helped |
| Initialize prototypes with K-Means rather than randomly (ablation) | PTaRL with FT-Transformer · Adult, Higgs, Helena, Jannis, ALOI, California housing | accuracy / RMSE: Adult: 0.863 -> 0.871; Higgs: 0.721 -> 0.738; Helena: 0.389 -> 0.397; Jannis: 0.729 -> 0.738; ALOI: 0.951 -> 0.97; Calif | Helped |
| Replace OT with Manhattan distance (ablation) | PTaRL with FT-Transformer · Adult, Higgs, Helena, Jannis, ALOI, California housing | accuracy / RMSE: PTaRL with OT: 0.871, 0.738, 0.397, 0.738, 0.97, 0.448; Manhattan: 0.859, 0.729, 0.382, 0.721, 0.953, 0.454 | Hurt |
| Replace OT with Euclidean distance (ablation) | PTaRL with FT-Transformer · Adult, Higgs, Helena, Jannis, ALOI, California housing | accuracy / RMSE: PTaRL with OT: 0.871, 0.738, 0.397, 0.738, 0.97, 0.448; Euclidean: 0.862, 0.713, 0.390, 0.724, 0.949, 0.459 | Hurt |
| Vary the orthogonalization loss weight from 0.25 to 0.2, 0.15, or 0.1 (ablation) | PTaRL with FT-Transformer · Adult, Higgs, Helena, Jannis, ALOI, California housing | accuracy / RMSE: 0.871, 0.738, 0.397, 0.738, 0.97, 0.448 -> 0.871, 0.740, 0.398, 0.737, 0.97, 0.449; 0.870, 0.739, 0.397, 0.737, 0.97, 0. | No clear effect |
| Remove the orthogonalization loss by setting its weight to 0.0 (ablation) | PTaRL with FT-Transformer · Adult, Higgs, Helena, Jannis, ALOI, California housing | accuracy / RMSE: 0.871, 0.738, 0.397, 0.738, 0.97, 0.448 -> 0.859, 0.722, 0.383, 0.725, 0.96, 0.453 | Hurt |
| Reduce the coordinates-diversifying loss weight from 0.25 to 0.15, 0.05, or 0.0 (ablation) | PTaRL with FT-Transformer · Adult, Higgs, Helena, Jannis, ALOI, California housing | accuracy / RMSE: 0.871, 0.738, 0.397, 0.738, 0.97, 0.448 -> 0.869, 0.737, 0.397, 0.736, 0.969, 0.451; 0.869, 0.737, 0.395, 0.734, 0.969, | Mixed |
Surprising findings
- Table 1 reports SNN on California housing at RMSE 0.898 without PTaRL versus 0.631 with it, a much larger gap than other settings.
Limitations
- Sensitivity to number of prototypes K and loss weights is only explored on a few settings in the appendix.
- Phase-2 training cost is substantially higher for AutoInt and FT-Transformer on larger datasets (e.g., up to ~23 s/epoch vs ~2 s/epoch for the base phase).
Experimental setup
Datasets: Adult Higgs Helena Jannis ALOI California housing
Benchmarks: Adult Higgs Helena Jannis ALOI California housing
Metrics: accuracy RMSE
Baselines: MLP DCNV2 SNN ResNet AutoInt FT-Transformer
Builds on: FT-Transformer optimal transport with entropic regularization K-Means clustering
Closest papers by meaning
- Representation Space Augmentation for Effective Self-Supervised Learning on Tabular Data (AAAI 2025 · similarity 0.90)
- ConTextTab: A Semantics-Aware Tabular In-Context Learner (NeurIPS 2025 · similarity 0.88)
- TRL-Bench: Standardizing Cross-Paradigm Representation-Level Evaluation of Tabular Encoders (NeurIPS 2026 · similarity 0.87)
- TabDPT: Scaling Tabular Foundation Models on Real Data (NeurIPS 2025 · similarity 0.86)
- TabSTAR: A Tabular Foundation Model for Tabular Data with Text Fields (NeurIPS 2025 · similarity 0.86)
- Hybrid Autoencoders for Tabular Data: Leveraging Model-Based Augmentation in Low-Label Settings (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.