Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning

ZEUS: Zero-shot Embeddings for Unsupervised Separation of Tabular Data

NeurIPS 2025 · Label-efficient and robust learning › Zero-/few-shot and meta-learning · approach: Transductive and semantic-dictionary zero-shot · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ZEUS, a zero-shot transformer (TabPFN-style) pre-trained on synthetic latent-mixture datasets to embed tabular data so that k-means can cluster it without per-dataset training.

Problem

Clustering tabular data is hard: similarity is dataset-dependent, and deep clustering methods are unstable and need per-dataset hyperparameter tuning without labels.

Approach

Pre-train a PFN-style transformer on synthetic Gaussian and neural-network-transformed mixture datasets with known labels, using a probabilistic centroid-based clustering loss with concentration and separation regularizers; at inference, a single forward pass produces embeddings that are clustered with k-means, with no labels or fine-tuning.

Key results

• ZEUS achieves mean ARI 57.43 on real OpenML datasets, best among methods (DEC 55.93, KM 55.54), and 86.33 on synthetic transformed data versus DEC 79.94.
• On synthetic Gaussian data ZEUS scores 89.03 ARI, second to k-means at 89.90.
• ZEUS obtains the best Brier score on synthetic transformed datasets (0.1796) and clustering time stays nearly constant with input size, close to k-means.

Measured effects reported in the paper

Surprising findings

• Off-the-shelf TabPFN and SCARF embeddings clustered with k-means perform poorly (e.g., SCARF 2.48 ARI on transformed synthetic data), showing supervised representations do not separate clusters well.
• Pre-training without the centroid-concentration regularizer gives weak clustering on synthetic data, so the regularizer is crucial for pre-training.

Limitations

• Inherits TabPFN limits: at most 30 input features (PCA beyond) and at most 2000 samples in experiments.
• Performance depends heavily on the synthetic prior; real datasets may be misspecified, as seen by the gap between real and synthetic results.

Experimental setup

Datasets: OpenML real datasets (34) synthetic Gaussian mixtures synthetic NN-transformed mixtures

Benchmarks: OpenML real datasets (34) Synthetic Gaussian (20) Synthetic Transformed (20)

Metrics: ARI average rank Brier score clustering time

Baselines: k-means GMM AE-KM AE-GMM DEC IDEC IDC G-CEALS TabPFN-Unsupervised SCARF

Builds on: TabPFN Prior-data fitted networks DEC IDEC Invertible ResNets

Closest papers by meaning

• Interpretable Deep Clustering for Tabular Data (ICML 2024 · similarity 0.85)
• TabArena: A Living Benchmark for Machine Learning on Tabular Data (NeurIPS 2025 · similarity 0.84)
• Zero-shot Meta-learning for Tabular Prediction Tasks with Adversarially Pre-trained Transformer (ICML 2025 · similarity 0.84)
• Hybrid Autoencoders for Tabular Data: Leveraging Model-Based Augmentation in Low-Label Settings (NeurIPS 2025 · similarity 0.83)
• TabM: Advancing tabular deep learning with parameter-efficient ensembling (ICLR 2025 · similarity 0.83)
• TabR: Tabular Deep Learning Meets Nearest Neighbors (ICLR 2024 · similarity 0.83)