Research map / Image generation / Autoregressive and latent generation

Tabular Foundation Model for Generative Modelling

NeurIPS 2026 · Image generation › Autoregressive and latent generation · approach: Generative latent space tooling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces TabFORGE, a tabular foundation generator that runs latent diffusion over frozen early-to-middle layers of a pretrained PFN (Real-TabPFN-2.5), with a decoder trained on denoised embeddings, and benchmarks it against 22 generators on 45 datasets.

Problem

Existing tabular foundation generators (autoregressive LLM/PFN-based, energy-based, and VAE-latent diffusion) do not consistently match strong dataset-specific generators, due to feature-order bias, local-only causal structure, and train-inference latent mismatch.

Approach

Build a frozen, causality-aware per-feature latent space from early-to-middle PFN layers, train a score-based (EDM) diffusion transformer over these embeddings, and then train a decoder directly on denoised embeddings so decoder training matches inference-time latents, with a lightweight per-dataset detokeniser for arbitrary categorical cardinality.

Key results

• Global utility improves 56.10% over TabICL (0.41 to 0.64) and 12.28% over TabSyn (0.57 to 0.64), the only foundation model surpassing the strongest dataset-specific generator.
• Local utility improves 15.28% over TabEBM (0.72 to 0.83), ranking second overall behind SMOTE.
• Fitting takes about 9.71% of TabSyn's training time and generation about 1.66% of TabEBM's generation time.

Surprising findings

• SMOTE achieves the highest local utility but shows signs of memorisation, outperforming real holdout data on beta-recall, authenticity and DCR.
• Deeper PFN encoder layers degrade generation quality, with deeper encoders performing worse than shallower ones.

Limitations

• Scale constrained by computational resources; no observed performance saturation during pretraining so larger pretraining may help.
• Pretraining uses only 43 datasets, limiting diversity of upstream data.
• Dependence on a frozen Real-TabPFN-2.5 encoder ties the representation to that PFN checkpoint.

Experimental setup

Datasets: TabArena TabStruct 43 pretraining datasets shared with Real-TabPFN-2.5

Benchmarks: TabArena TabStruct

Metrics: global utility (structural fidelity) local utility (ML efficacy) shape trend alpha-precision beta-recall authenticity DCR score

Baselines: TabPFN TabICL TabEBM CTSyn GReaT TabSyn SMOTE TabDDPM

Builds on: TabPFN EDM TabSyn TabICL

Closest papers by meaning

• Understanding the Surprising Generalization Properties of Tabular Foundation Models (NeurIPS 2026 · similarity 0.91)
• TabDPT: Scaling Tabular Foundation Models on Real Data (NeurIPS 2025 · similarity 0.90)
• CTSyn: A Foundation Model for Cross Tabular Data Generation (ICLR 2025 · similarity 0.90)
• TabNAT: A Continuous-Discrete Joint Generative Framework for Tabular Data (ICML 2025 · similarity 0.89)
• TabSTAR: A Tabular Foundation Model for Tabular Data with Text Fields (NeurIPS 2025 · similarity 0.89)
• MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image (NeurIPS 2026 · similarity 0.88)