Research map / Federated learning and privacy / Differential privacy

Breaking the Quality–Privacy Tradeoff in Tabular Data Generation via In-Context Learning

NeurIPS 2026 · Federated learning and privacy › Differential privacy · approach: Differentially private synthetic data · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DiffICL, a latent diffusion model pretrained with an in-context learning objective (context/query split over ~4,130 permuted tabular datasets) that generates synthetic tables conditioned on a small real context, claiming better quality-privacy tradeoff and useful augmentation on 14 datasets.

Problem

Existing tabular generative models in small-data regimes trade data quality against memorization (privacy leakage), with no favorable Pareto frontier.

Approach

Formulate tabular generation as in-context learning: pretrain a conditional latent diffusion model to generate a query set from a disjoint context set across many datasets, so memorizing the context does not lower loss; at inference the real data serves as context over LimiX-encoded latents with per-dataset MLP decoders.

Key results

• On 14 datasets, DiffICL achieves the best downstream utility and DCROverfit across all four learners (e.g., XGB 74.0 vs 42.8 for best-other CTGAN-range baselines; DCR 76.1 vs 73.2 TabDiff).
• In the augmentation setting (real + 2500 synthetic), DiffICL improves over real-data baseline (XGB 79.8 vs 76.9 on 14 datasets) while all other generators degrade performance.
• Ablation: removing cross-dataset pretraining (DiffICL(S)) drops synthetic-only XGB performance from 74.0 to 57.0.

Surprising findings

• Distribution-based quality metrics (Shape, Trend, IP, IR) correlate weakly (Pearson below 0.6) with downstream ML efficiency.
• Among evaluated deep generators, only DiffICL improves downstream performance when augmenting real data; all other methods degrade it.

Limitations

• GReaT failed to generate valid samples on 9 of 14 datasets, limiting comparison to subsets.
• Pretraining cost (32 RTX 4090 GPUs, 20,000 epochs) is excluded from efficiency analysis.
• Privacy evaluated only with DCROverfit on the end-to-end pipeline; weights and code to be released only upon acceptance (stated intention).

Experimental setup

Datasets: kc2 wdbc ilpd diabetes vehicle mfeat-zernike kr-vs-kp red wine auction verification abalone white wine wind

Benchmarks: kc2 wdbc ilpd diabetes vehicle mfeat-zernike kr-vs-kp red wine auction verification abalone white wine wind

Metrics: AUC R^2 DCROverfit Shape Trend IP_alpha IR_beta

Baselines: TVAE CTGAN CTABGAN+ tabDDPM TabDiff TabSYN GReaT

Builds on: LimiX EDM TabSYN TabPFN

Closest papers by meaning

• HARMONIC: Harnessing LLMs for Tabular Data Synthesis and Privacy Protection (NeurIPS 2024 · similarity 0.89)
• CuTS: Customizable Tabular Synthetic Data Generation (ICML 2024 · similarity 0.88)
• Do You Really Need Public Data? Surrogate Public Data for Differential Privacy on Tabular Data (NeurIPS 2025 · similarity 0.88)
• TabDiff: a Mixed-type Diffusion Model for Tabular Data Generation (ICLR 2025 · similarity 0.86)
• Tabular Foundation Model for Generative Modelling (NeurIPS 2026 · similarity 0.86)
• Masked Language Modeling Becomes Conditional Density Estimation for Tabular Data Synthesis (AAAI 2025 · similarity 0.86)