Research map / Image generation / Fast diffusion sampling
Continuous Diffusion for Mixed-Type Tabular Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CDTD, a continuous diffusion model for mixed-type tabular data that unifies continuous and categorical features under Gaussian noise (categorical via embeddings and score interpolation), with loss calibration and adaptive per-type/feature noise schedules.
Problem
Diffusion models for mixed-type tabular data mismatch noise processes and losses across continuous and categorical features, use discrete multinomial diffusion that scales poorly to high-cardinality categories, and reuse image-domain noise schedules.
Approach
Combine denoising score matching for continuous features with score interpolation for categorical embeddings so all features share a Gaussian noise process; calibrate type losses so they are homogeneous at t=1, initialize output layers accordingly, and learn adaptive domain-adapted logistic timewarping noise schedules per type or feature.
Key results
- CDTD (per type) achieves the best average rank on RMSE (2.6), F1 (2.3), AUC (2.7), correlation L2 distance (2.0) and detection score (1.6) across datasets.
- Data type homogenization (Config. B vs. A) substantially improves RMSE (0.090 to 0.069) and detection score (0.762 to 0.739) in the ablation.
- Per-type noise schedules mostly outperform per-feature and single schedules, and CDTD trains faster than CoDi and TabSyn-style baselines on high-cardinality data.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use type-specific rather than a single noise schedule | CDTD · 11-dataset average rank | L2 correlation distance rank: 3.4_{\pm 1.1} -> 2.0_{\pm 0.8} | Helped |
| use type-specific rather than per-feature noise schedules | CDTD · 11-dataset average rank | Detection score rank: 3.3_{\pm 1.1} -> 1.6_{\pm 1.0} | Helped |
| add data-type homogenization (loss calibration, improved initialization, and time-dependent normalization) (ablation) | CDTD Config A · Ablation study median | RMSE absolute difference: 0.090 -> 0.069 | Helped |
| add data-type homogenization (loss calibration, improved initialization, and time-dependent normalization) (ablation) | CDTD Config A · Ablation study median | Detection score: 0.762 -> 0.739 | Helped |
| replace the piecewise-linear schedule with the proposed functional form using uniform initialization (ablation) | CDTD Config B · Ablation study median | Detection score: 0.739 -> 0.774 | Hurt |
| add per-type noise schedules (ablation) | CDTD Config C · Ablation study median | RMSE absolute difference: 0.060 -> 0.058 | Helped |
| initialize schedules to emphasize low noise levels (ablation) | CDTD Config D · Ablation study median | F1 absolute difference: 0.006 -> 0.009 | Hurt |
| use per-feature rather than type-specific noise schedules (ablation) | CDTD · 11-dataset average rank | L2 correlation distance rank: 2.0_{\pm 0.8} -> 2.8_{\pm 1.5} | Hurt |
Surprising findings
- TabSyn, a latent-space diffusion model, performs considerably worse than CDTD and often TabDDPM, which operate in data space.
- Per-feature noise schedules decrease sample quality relative to per-type schedules, as they impose too many constraints on the model.
Limitations
- Adds hyperparameters that can be costly to tune, though per-type schedules and defaults work well across datasets.
- Privacy risk must be checked when generating from sensitive sources; DCR only indicates no issues on benchmark data.
- May be outperformed by specialized models for time-series; imputation adaptation left for future work.
Experimental setup
Datasets: acsincome adult bank beijing churn covertype default diabetes lending news nmes
Benchmarks: acsincome adult bank beijing churn covertype default diabetes lending news nmes
Metrics: RMSE difference F1 difference AUC difference L2 distance of correlation matrices detection score JSD Wasserstein distance distance to closest record training and sampling time
Baselines: SMOTE ARF CTGAN TVAE TabDDPM CoDi TabSyn
Builds on: EDM TabDDPM score interpolation multinomial diffusion TabSyn
Closest papers by meaning
- TabDiff: a Mixed-type Diffusion Model for Tabular Data Generation (ICLR 2025 · similarity 0.92)
- Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Space (ICLR 2024 · similarity 0.90)
- CTSyn: A Foundation Model for Cross Tabular Data Generation (ICLR 2025 · similarity 0.88)
- TabNAT: A Continuous-Discrete Joint Generative Framework for Tabular Data (ICML 2025 · similarity 0.88)
- Exponential Family Variational Flow Matching for Tabular Data Generation (ICML 2025 · similarity 0.88)
- Understanding and Mitigating Memorization in Diffusion Models for Tabular Data (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.