Research map / Image generation / Autoregressive and latent generation
CTSyn: A Foundation Model for Cross Tabular Data Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CTSyn, a diffusion-based tabular foundation model that pairs a schema-conditioned VAE autoencoder with a conditional latent diffusion transformer, pre-trained on 86 OpenTab tables for cross-table synthetic data generation.
Problem
Synthetic tabular data generators fail to help downstream models when real data is scarce, and existing transferable tabular learners either lack a generative backbone or treat tables as unstructured text, so heterogeneous table schemas cannot be shared across datasets.
Approach
A cross-tabular VAE embeds rows (column-name and value embeddings from a text embedding model) into a unified latent space and decodes cell values guided by column-name and LLM-generated metadata embeddings, with a conditional latent diffusion transformer generating latents conditioned on the table schema.
Key results
- CTSyn achieves the highest average correlation similarity (0.95) and recall (0.075) among compared synthesizers in statistical fidelity.
- CTSyn outperforms all baselines in low-data (N<=100) classification utility, e.g., 0.79 F1 at 30 shots vs 0.77 for TabDDPM and 0.76 for TabSyn.
- Pre-training improves CTSyn's synthetic F1 from 0.80 to 0.84 and reduces RMSE from 0.17 to 0.13, while GReaT and TabSyn show mostly negative pre-training effects.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| pre-train CTSyn on cross-table data (ablation) | CTSyn · Benchmark datasets | Corr: 0.90 -> 0.95 | Helped |
| pre-train CTSyn on cross-table data (ablation) | CTSyn · Benchmark datasets | Synth F1: 0.80 -> 0.84 | Helped |
| pre-train CTSyn on cross-table data (ablation) | CTSyn · Benchmark datasets | PCT: 0.90 -> 0.97 | Helped |
| remove metadata embedding from decoder guidance (ablation) | CTSyn autoencoder · Unseen Columns | MSE: 0.0063 -> 0.0068 | Hurt |
| remove column-name embeddings (ablation) | CTSyn autoencoder · Unseen Columns | MSE: 0.0063 -> 0.0082 | Hurt |
| replace column-name and metadata guidance with positional encoding (ablation) | CTSyn autoencoder · Permuted Columns | MSE: 0.0005 -> 0.08 | Hurt |
| replace column-name and metadata guidance with positional encoding (ablation) | CTSyn autoencoder · Permuted Columns | Acc: 0.94 -> 0.67 | Hurt |
| use uninformative English-letter column names at fine-tuning (ablation) | CTSyn · Benchmark datasets | Corr: 0.95 (0.02) -> 0.93 (0.03) | Hurt |
| use uninformative English-letter column names at fine-tuning (ablation) | CTSyn · Benchmark datasets | Recall: 0.075 (0.006) -> 0.070 (0.007) | Hurt |
| pre-train TabSyn on cross-table data (ablation) | TabSyn · Benchmark datasets | Synth F1: 0.84 -> 0.75 | Hurt |
| pre-train GReaT on cross-table data (ablation) | GReaT · Benchmark datasets | Synth F1: 0.83 -> 0.79 | Hurt |
| use CTSyn rather than TabSyn | synthetic-data generation · Benchmark datasets | Recall: 0.003 -> 0.075 | Helped |
Surprising findings
- Pre-training degraded GReaT and TabSyn on several metrics, suggesting baseline generators fail to transfer knowledge across tables.
- Released-free baselines AIM and PATE-CTGAN obtain high PCT and authenticity scores but poor fidelity and utility, reflecting a privacy-utility trade-off.
Limitations
- Only numerical and categorical columns in main experiments; text and date-time support is only preliminary in appendix.
- Transfer beyond simple pre-training/fine-tuning is not explored, and utility gains diminish as data size grows.
- Relies on clean, large-scale tabular data with semantically meaningful category names.
Experimental setup
Datasets: OpenTab (filtered, 86 tables, 5.01M rows) for pre-training 11 downstream benchmarks (Faults, Wilt, HTRU2, News, Bean, Obesity, Titanic, Insurance, Abalone, Shoppers, Indian Liver
Benchmarks: 11 downstream tabular datasets (classification and regression) in few-shot settings of 30-500 rows and full data
Metrics: Shape (KS/TVD) Correlation Precision Recall F1 RMSE PCT Authenticity MSE and accuracy for reconstruction
Baselines: SMOTE CTGAN TVAE AIM PATE-CTGAN TabDDPM TabSyn GReaT
Builds on: TabSyn Perceiver Resampler beta-VAE DDPM GReaT
Closest papers by meaning
- Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Space (ICLR 2024 · similarity 0.90)
- Tabular Foundation Model for Generative Modelling (NeurIPS 2026 · similarity 0.90)
- TabNAT: A Continuous-Discrete Joint Generative Framework for Tabular Data (ICML 2025 · similarity 0.89)
- TabDiff: a Mixed-type Diffusion Model for Tabular Data Generation (ICLR 2025 · similarity 0.89)
- Continuous Diffusion for Mixed-Type Tabular Data (ICLR 2025 · similarity 0.88)
- ClavaDDPM: Multi-relational Data Synthesis with Cluster-guided Diffusion Models (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.