Research map / Label-efficient and robust learning / OOD and anomaly detection

FlexTab: A Flexible Encoder-Decoder Architecture for In-Context Learning Across Diverse Tabular Tasks

NeurIPS 2026 · Label-efficient and robust learning › OOD and anomaly detection · approach: Deep one-class and tabular anomaly detection · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FlexTab, an encoder-decoder tabular in-context learner with a shared target-agnostic row encoder and task-specific decoders covering classification, regression, anomaly detection, clustering, matching, and relational entity classification.

Problem

Existing tabular in-context learners (TabPFN, TabICL, Mitra, ConTextTab) produce target-aware representations and are largely limited to classification and regression, so they cannot natively support other tabular tasks.

Approach

A single task-agnostic encoder produces target-agnostic row embeddings (learned aggregation of [ROW] tokens across all encoder layers), which are consumed by separate task-specific decoders that attend to them via cross-attention; the encoder is pretrained on classification/regression and frozen while other decoders are trained on top.

Key results

• On CARTE, FlexTab [bag=8] ranks 1.6 (acc 77.3, R2 74.4), best among compared models in that group; overall mean rank 2.7 excluding AutoGluon.
• On matching, FlexTab-Match achieves average F1 91.2 vs 90.2 for Hybrid DeepMatcher and 84.9 for DITTO across five datasets.
• On RelBench entity classification, FlexTab-Multi mean AUROC 70.93, on par with Relational Transformer (70.65) but below KumoRFM-1 (76.72).

Surprising findings

• Using only the last encoder layer output led to training collapse after about 2M steps; learned per-layer projections of [ROW] tokens were needed.
• Simple n-gram hash embedding similarity outperforms FlexTab-based matching on some datasets, and tabular ICL models recast as single-table matching perform poorly.

Limitations

• Compressing each row into a single embedding may create an information bottleneck for wide tables or fine-grained cell-level reasoning.
• Contamination found: 8 evaluation tables exactly appear in pretraining corpus and 25 have matching pretraining tables by estimate.
• Clustering evaluation deferred due to training and validation instabilities; clustering results not reported in main text.

Experimental setup

Datasets: internal ~300k unlabeled real-world table corpus TabArena-Lite TALENT-Tiny CARTE TextTab ADBench Febrl4 Fodors-Zagats Bikes eBooks Movies RelBench

Benchmarks: TabArena-Lite TALENT-Tiny CARTE TextTab ADBench Febrl4 Fodors-Zagats Bikes eBooks Movies RelBench

Metrics: accuracy R2 (soft-clipped) mean rank AUROC F1 mean AUROC

Baselines: TabPFNv2.6 TabICLv2 ConTextTab CatBoost RealMLP AutoGluon TACTIC KNN Relational Transformer Griffin RDBLearn KumoRFM-1

Builds on: ConTextTab TabPFN TabICL Mitra

Closest papers by meaning

• ConTextTab: A Semantics-Aware Tabular In-Context Learner (NeurIPS 2025 · similarity 0.88)
• SwitchTab: Switched Autoencoders Are Effective Tabular Learners (AAAI 2024 · similarity 0.88)
• Enhancing Tabular Learners with Context-Aware Semantic Embeddings (NeurIPS 2026 · similarity 0.87)
• TRL-Bench: Standardizing Cross-Paradigm Representation-Level Evaluation of Tabular Encoders (NeurIPS 2026 · similarity 0.87)
• Understanding the Surprising Generalization Properties of Tabular Foundation Models (NeurIPS 2026 · similarity 0.87)
• TabDPT: Scaling Tabular Foundation Models on Real Data (NeurIPS 2025 · similarity 0.86)