Research map / Label-efficient and robust learning / Recognition and long-tail
TabReD: Analyzing Pitfalls and Filling the Gaps in Tabular Deep Learning Benchmarks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces TabReD, eight industry-grade tabular datasets with time-based splits and rich feature engineering, and benchmarks tabular DL methods, finding MLP with numerical embeddings and GBDTs perform best while many recent methods fail to transfer.
Problem
Existing tabular deep learning benchmarks under-represent two industrial conditions: gradual temporal distribution drift (requiring time-based splits) and feature-rich data from extensive feature engineering, so reported method gains may not transfer to deployment.
Approach
Audit ~100 public tabular datasets for leakage, synthetic/untraceable and non-tabular issues, then build TabReD from Kaggle competitions and new industrial datasets that all have timestamps, time-based train/val/test splits, and many features, and re-evaluate recent tabular DL methods on them and compare rankings against random splits.
Key results
- MLP with numerical embeddings (MLP-PLR) and GBDTs (XGBoost, LightGBM, CatBoost) achieve the best average ranks on TabReD, with MLP-PLR ensemble at 2.4 and XGBoost at 2.9.
- On time-based splits, XGBoost's performance margin over MLPs shrinks compared to random splits, and random splits give more optimistic scores.
- Retrieval-based models (TabR, ModernNCA) and long training recipes with augmentation do not transfer to TabReD, while ensembles and numerical embeddings do.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add numerical feature embeddings (PLR) to MLP | MLP · TabReD datasets | average rank: 5.0 ± 1.8 -> 3.8 ± 1.4 | Helped |
| ensemble MLP models | MLP · TabReD datasets | average rank: 5.0 ± 1.8 -> 4.1 ± 2.0 | Helped |
| ensemble MLP-PLR models | MLP-PLR · TabReD datasets | average rank: 3.8 ± 1.4 -> 2.4 ± 1.5 | Helped |
| use MLP aug. long-training and augmentation recipe | MLP · TabReD datasets | average rank: 5.0 ± 1.8 -> 6.0 ± 2.0 | Hurt |
| add reconstruction loss to MLP aug. | MLP · TabReD datasets | ROC AUC: 0.8449 -> 0.7453 | Hurt |
| use retrieval-based TabR | MLP baseline · TabReD datasets | average rank: 5.0 ± 1.8 -> 6.0 ± 2.2 | Hurt |
| use ModernNCA retrieval model | MLP baseline · TabReD datasets | average rank: 5.0 ± 1.8 -> 5.6 ± 1.6 | Mixed |
| add DeepCORAL temporal-shift adaptation | MLP · TabReD datasets | average rank: 1.0 ± 0.0 -> 1.4 ± 0.7 | No clear effect |
| add Deep Feature Reweighting | MLP · TabReD datasets | average rank: 1.0 ± 0.0 -> 1.4 ± 0.5 | No clear effect |
| evaluate with time-based rather than random splits | tabular ML models · TabReD datasets | model ranking | Mixed |
| evaluate XGBoost on time-shifted rather than random test splits | XGBoost · Sberbank Housing, Cooking Time, Delivery ETA, Weather, Ecom Offers and Quote Conversion | performance margin over MLP | Hurt |
| use TabR-S instead of MLP | MLP baseline · Cooking Time and Homesite Insurance | performance | Helped |
Surprising findings
- Retrieval-based models and improved training recipes that help on academic benchmarks underperform simple MLPs on TabReD.
- Evaluating on random splits instead of time-based splits changes method rankings and inflates XGBoost's margin.
Limitations
- Benchmark is biased toward industry applications with large samples and temporal drift, and does not cover medicine, science or social data.
- Newly introduced datasets are anonymized and lack precise feature names, limiting use with LLMs; full data release deferred until acceptance.
- Large datasets were randomly subsampled for tuning, and Trompt was reproduced by the authors without official code.
Experimental setup
Datasets: Sberbank Housing Homesite Insurance Ecom Offers HomeCredit Default Cooking Time Delivery ETA Maps Routing Weather
Benchmarks: TabReD
Metrics: ROC AUC RMSE average rank
Baselines: XGBoost LightGBM CatBoost RandomForest Linear MLP SNN DCNv2 ResNet FT-Transformer MLP-PLR Trompt
Builds on: Gorishniy et al. tabular DL benchmark TabR ModernNCA Trompt DeepCORAL Deep Feature Reweighting
Closest papers by meaning
- TabArena: A Living Benchmark for Machine Learning on Tabular Data (NeurIPS 2025 · similarity 0.90)
- TabM: Advancing tabular deep learning with parameter-efficient ensembling (ICLR 2025 · similarity 0.90)
- TabR: Tabular Deep Learning Meets Nearest Neighbors (ICLR 2024 · similarity 0.89)
- A Data-Centric Perspective on Evaluating Machine Learning Models for Tabular Data (NeurIPS 2024 · similarity 0.89)
- TabFSBench: Tabular Benchmark for Feature Shifts in Open Environments (ICML 2025 · similarity 0.89)
- TabPrep: Closing the Feature Engineering Gap in Tabular Benchmarks (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.