Research map / Image generation / Autoregressive and latent generation

TabNAT: A Continuous-Discrete Joint Generative Framework for Tabular Data

ICML 2025 · Image generation › Autoregressive and latent generation · approach: Next-scale and token autoregressive images

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TabNAT, a bidirectional masked Transformer that nests a conditional diffusion head for continuous columns and categorical cross-entropy heads for discrete columns, enabling order-agnostic tabular generation and imputation.

Problem

Autoregressive models struggle with heterogeneous (continuous and discrete) tabular columns and with permutation-invariant, flexible generation orders needed for tasks like missing data imputation.

Approach

Combines a masked bidirectional Transformer (predicting conditioning embeddings for masked columns in parallel, with random training order) with a per-column diffusion loss for continuous columns and cross-entropy for discrete columns, so sampling can follow any column order.

Key results

• Averaged over ten datasets, TabNAT reduces Marginal error to 1.05% and Joint error to 1.71%, versus 1.36% and 1.91% for the second-best TabSyn, a 22.8% and 10.5% relative improvement.
• TabNAT improves MLE AUC/RMSE gap by 39.5% over the best baseline in the reported comparison, and is best on five of six fidelity metrics; SMOTE wins on beta-Recall.
• On 30% MCAR imputation across five heterogeneous datasets, TabNAT achieves comparable or better MAE and accuracy than KNN, GRAPE, MOT, HyperImpute and Remasker.

Measured effects reported in the paper

Surprising findings

• Interpolation baseline SMOTE achieves the best beta-Recall and produces samples too close to training data, which hurts privacy (DCR).
• Removing the diffusion loss (using MSE instead) causes the largest drop in fidelity, larger than removing random order.

Limitations

• Imputation evaluated only in out-of-sample setting with MCAR 30% missingness; TabNAT is not primarily designed for imputation.
• Experiments limited to tabular datasets up to moderate size; discrete columns use independent per-column heads with no explicit joint modeling among discrete tokens beyond attention.

Experimental setup

Datasets: California Housing Letter Car Nursery Adult Default Shoppers Magic News Beijing

Benchmarks: Adult California Letter Car Nursery Default Magic News Shoppers Beijing

Metrics: Marginal Joint alpha-Precision beta-Recall C2ST JSD MLE AUC/RMSE gap DCR privacy MAE accuracy

Baselines: SMOTE TVAE GOGGLE CTGAN GReaT STaSy CoDi TabDDPM TabSyn DP-TBART Tab-MT KNN

Builds on: TabSyn TabDDPM RePaint EDM

Closest papers by meaning

• TabDiff: a Mixed-type Diffusion Model for Tabular Data Generation (ICLR 2025 · similarity 0.91)
• Tabular Foundation Model for Generative Modelling (NeurIPS 2026 · similarity 0.89)
• CTSyn: A Foundation Model for Cross Tabular Data Generation (ICLR 2025 · similarity 0.89)
• Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Space (ICLR 2024 · similarity 0.88)
• Continuous Diffusion for Mixed-Type Tabular Data (ICLR 2025 · similarity 0.88)
• TabSDS: a Lightweight, Fully Non-Parametric, and Model Free Approach for Generating Synthetic Tabular Data (ICML 2025 · similarity 0.88)