Research map / Label-efficient and robust learning / Recognition and long-tail

TabEBM: A Tabular Data Augmentation Method with Distinct Class-Specific Energy-Based Models

NeurIPS 2024 · Label-efficient and robust learning › Recognition and long-tail · approach: Group-robust classification · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TabEBM, which builds one energy-based generator per class by reinterpreting TabPFN logits on a surrogate binary task, and uses it for training-free tabular data augmentation and generation.

Problem

Tabular data augmentation for small datasets: existing generators (joint or shared class-conditional models) overfit, break label distribution, and often fail to improve classifier accuracy over real-data-only training.

Approach

Train a separate surrogate binary classifier per class (class data vs. negative samples placed at hypercube corners far from data) using pre-trained TabPFN as a training-free in-context learner; reinterpret its logits as an energy function, then sample with SGLD. Class-specific EBMs keep the label distribution and avoid shared-model class confusion.

Key results

• TabEBM achieves the best average accuracy rank (1.21) for data augmentation across eight datasets and sample sizes, versus 3.15 for TabPFGen and 3.03 for SMOTE.
• TabEBM is the only generator that consistently improves over the real-data baseline across sample sizes, e.g. average balanced accuracy 64.97 vs 62.07 for baseline in the ablation subset.
• TabEBM yields higher statistical fidelity and privacy trade-offs, and is the only generator that surpasses the baseline in train-on-synthetic, test-on-real setting.

Measured effects reported in the paper

Surprising findings

• Most modern benchmark generators (TVAE, CTGAN, NFLOW, TabDDPM, ARF, GOGGLE) underperform even the real-data baseline in the low-sample-size regime.
• Setting synthetic size equal to real size (N_syn = N_real) leads to highly unstable results, motivating a fixed N_syn=500.

Limitations

• Inherits TabPFN limitations, notably scaling to larger numbers of features.
• Performance gains decrease as sample size increases.
• Evaluated only with TabPFN as the surrogate binary classifier; other classifiers left for future work.

Experimental setup

Datasets: protein fourier biodeg steel stock energy collins texture UCI leakage-free datasets

Benchmarks: protein fourier biodeg steel stock energy collins texture

Metrics: balanced accuracy average accuracy rank ADTM inverse KL divergence KS test p-value chi-squared test p-value DCR delta-presence

Baselines: SMOTE TVAE CTGAN NFLOW TabDDPM ARF GOGGLE TabPFGen Baseline (real data only)

Builds on: TabPFN Energy-Based Models Stochastic Gradient Langevin Dynamics TabPFGen SMOTE

Closest papers by meaning

• Synthetic Tabular Data Generation for Imbalanced Classification: The Surprising Effectiveness of an Overlap Class (AAAI 2025 · similarity 0.88)
• TabSDS: a Lightweight, Fully Non-Parametric, and Model Free Approach for Generating Synthetic Tabular Data (ICML 2025 · similarity 0.88)
• Tabular Foundation Model for Generative Modelling (NeurIPS 2026 · similarity 0.87)
• Hybrid Autoencoders for Tabular Data: Leveraging Model-Based Augmentation in Low-Label Settings (NeurIPS 2025 · similarity 0.87)
• TabDPT: Scaling Tabular Foundation Models on Real Data (NeurIPS 2025 · similarity 0.86)
• TabICL: A Tabular Foundation Model for In-Context Learning on Large Data (ICML 2025 · similarity 0.86)