Research map / Image generation / Autoregressive and latent generation
Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Space
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TabSyn, which encodes mixed-type tabular rows into a VAE latent space with a Transformer tokenizer/encoder and then trains a score-based diffusion model with linear noise schedule there, achieving better synthetic data quality and fewer sampling steps.
Problem
Synthesizing mixed-type tabular data with diffusion models is hard because of varied column distributions and the mix of continuous and categorical features, which standard Gaussian diffusion handles poorly; existing diffusion methods are also slow to sample.
Approach
Map tabular rows into a unified continuous latent space with a Transformer-based VAE (per-column tokenizer, adaptive beta-VAE weighting), then run a VE-type score-based diffusion with linear noise schedule sigma(t)=t in that latent space, which permits fewer than 20 reverse steps.</ยcore_idea>
Key results
- Column-wise density error averaged 1.08% for TabSyn vs. 7.72% for the best prior baseline (STaSy), an 86% reduction.
- Pair-wise correlation error averaged 1.73% for TabSyn, a 67.6% reduction over the best baseline (TabDDPM at 5.34%).
- TabSyn generates high-quality samples in fewer than 20 reverse steps, while STaSy needs 50-200 NFEs and TabDDPM needs about 1,000.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TabSyn instead of the strongest generative baseline | Tabular data synthesis · Six datasets (average) | column-wise density error rate (%): 86.0\% \downarrow | Helped |
| Use TabSyn instead of the strongest baseline | Tabular data synthesis · Six datasets (average) | pair-wise column correlation error rate (%): 67.6\% \downarrow | Helped |
| Use TabSyn instead of baseline methods | Machine learning efficiency evaluation · Adult, Default, Shoppers, Magic, Beijing, News | AUC (classification) and RMSE (regression) | Helped |
| Use scheduled β instead of a constant β (ablation) | TabSyn VAE · Synthetic data quality: single-column density and pair-wise column correlation | error rate | Helped |
| Use linear noise levels, σ(t) = t (ablation) | TabSyn diffusion process · Single-column density and pair-wise column correlation | error rate: fewer than 20 NFEs | Helped |
| Replace the VAE latent representation with one-hot categorical encodings (ablation) | TabSyn · Synthetic tabular data quality | error rate | Hurt |
| Replace TabSyn's diffusion process with DDPM in latent space (ablation) | TabSyn · Synthetic tabular data quality | error rate | Hurt |
| Apply DDPM in the VAE latent space rather than the data space (ablation) | TabDDPM · Synthetic tabular data quality | error rate | Helped |
| Reduce diffusion sampling steps (ablation) | TabSyn · Single-column density and pair-wise column correlation | synthetic data quality: fewer than 20 NFEs; 50-200 NFEs; 1,000 NFEs | Mixed |
Surprising findings
- One-hot encoding of categorical columns with a continuous diffusion model (TabSyn-OneHot) gave the worst performance, indicating categorical columns should not be treated as continuous features.
- GReaT performs much worse on pair-wise correlation than on column-wise density, suggesting autoregressive LMs struggle with joint distributions.
Limitations
- Evaluation limited to six UCI datasets and unconditional generation.
- The VAE latent dimension d=4 per column and other design choices are tuned per setup; generality beyond these datasets is untested.
Experimental setup
Datasets: Adult Default Shoppers Magic Beijing News
Benchmarks: Adult Default Shoppers Magic Beijing News
Metrics: KST TVD Pearson correlation difference contingency similarity AUC RMSE C2ST alpha-precision beta-recall DCR
Baselines: SMOTE CTGAN TVAE GOGGLE GReaT STaSy CoDi TabDDPM
Builds on: TabDDPM STaSy beta-VAE EDM (score-based SDE formulation) DDIM
Closest papers by meaning
- CTSyn: A Foundation Model for Cross Tabular Data Generation (ICLR 2025 · similarity 0.90)
- TabDiff: a Mixed-type Diffusion Model for Tabular Data Generation (ICLR 2025 · similarity 0.90)
- Continuous Diffusion for Mixed-Type Tabular Data (ICLR 2025 · similarity 0.90)
- TabNAT: A Continuous-Discrete Joint Generative Framework for Tabular Data (ICML 2025 · similarity 0.88)
- Tabular Foundation Model for Generative Modelling (NeurIPS 2026 · similarity 0.88)
- ClavaDDPM: Multi-relational Data Synthesis with Cluster-guided Diffusion Models (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.