Research map / Label-efficient and robust learning / Test-time adaptation
A Data-Centric Perspective on Evaluating Machine Learning Models for Tabular Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a data-centric evaluation framework for tabular ML built on 10 Kaggle competitions with expert feature-engineering and test-time adaptation pipelines, showing model rankings and gaps change substantially once dataset-specific preprocessing is applied.
Problem
Model-centric evaluations of tabular ML models use standardized preprocessing and lack external performance references, which limits their validity for real-world pipelines that rely on dataset-specific feature engineering and face temporal distribution shifts.
Approach
Evaluate tabular models under three preprocessing pipelines (standardized, expert feature engineering, and expert feature engineering with test-time adaptation) derived from implemented top Kaggle solutions, and report results as percentiles on Kaggle private leaderboards as an external reference.
Key results
- Model rankings change considerably between standardized and expert preprocessing, with low Spearman correlations and shrinking performance gaps between models after feature engineering.
- For SCTP, the best single model's leaderboard percentile rises from 0.518 (standardized) to 0.962 (expert FE) and 0.992 (FE with TTA).
- Test-time feature engineering consistently improves single-model performance, and for three datasets top-1% leaderboard rank was not achieved without test-time adaptation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use OHE instead of model-default categorical treatment | Tree-based models · PSSDP | performance: 0.69 -> 0.99; 0.54 -> 0.94; 0.71 -> 0.97 | Helped |
| Use target encoding instead of model-default categorical treatment | Tree-based models · BPCCM | performance: 0.4 -> 0.99; 0.35 -> 0.99 | Helped |
| Use target encoding instead of model-default categorical treatment | CatBoost · BPCCM | performance: 1.0 -> 1.0 | No clear effect |
| Add expert feature engineering | Best single model · AEAC | leaderboard position: 0.953 -> 0.937 | Hurt |
| Add expert feature engineering | Best single model · OGPCC | leaderboard position: 0.896 -> 0.871 | Hurt |
| Add expert feature engineering | Best single model · SCS | leaderboard position: 0.945 -> 0.953 | Helped |
| Add expert feature engineering | Best single model · SCTP | leaderboard position: 0.518 -> 0.962 | Helped |
| Add expert feature engineering | Best single model · IFD | leaderboard position: 0.662 -> 0.988 | Helped |
| Add expert feature engineering | Best single model · PSSDP | leaderboard position: 0.656 -> 0.994 | Helped |
| Add test-time adaptation after expert feature engineering | Best single model · AEAC | leaderboard position: 0.937 -> 0.991 | Helped |
| Add test-time adaptation after expert feature engineering | Best single model · OGPCC | leaderboard position: 0.871 -> 0.923 | Helped |
| Add test-time adaptation after expert feature engineering | Best single model · SCS | leaderboard position: 0.953 -> 0.975 | Helped |
Surprising findings
- CatBoost's superiority in standardized preprocessing largely vanishes once other models receive dataset-specific feature engineering.
- AutoGluon was sometimes outperformed by single models it contains, and did not always benefit from feature engineering to the same extent.
Limitations
- Single test set per dataset and no error bars due to compute cost, so small differences are uncertain.
- Leaderboard percentiles vary across competitions and may not reflect large differences in the actual metric; some submissions are non-expert.
- Datasets are mostly finance-domain and from North America and Europe.
Experimental setup
Datasets: MBGM SVPC AEAC OGPCC SCS BPCCM SCTP HQC IFD PSSDP
Benchmarks: Kaggle private leaderboards (10 competitions)
Metrics: leaderboard percentile task metric (appendix) Spearman correlation
Baselines: XGBoost LightGBM CatBoost ResNet FTTransformer MLP-PLR GRANDE AutoGluon
Builds on: TPE (tree-structured Parzen estimator) AdamW Optuna
Closest papers by meaning
- TabPrep: Closing the Feature Engineering Gap in Tabular Benchmarks (NeurIPS 2026 · similarity 0.91)
- TabFSBench: Tabular Benchmark for Feature Shifts in Open Environments (ICML 2025 · similarity 0.90)
- TabReD: Analyzing Pitfalls and Filling the Gaps in Tabular Deep Learning Benchmarks (ICLR 2025 · similarity 0.89)
- TabArena: A Living Benchmark for Machine Learning on Tabular Data (NeurIPS 2025 · similarity 0.89)
- Large Scale Transfer Learning for Tabular Data via Language Modeling (NeurIPS 2024 · similarity 0.88)
- Revisiting Nearest Neighbor for Tabular Data: A Deep Tabular Baseline Two Decades Later (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.