Research map / Label-efficient and robust learning / Recognition and long-tail
Why do tree-based models still outperform deep learning on typical tabular data?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use tree-based models rather than neural networks | Medium-sized tabular datasets · 45-dataset benchmark | test accuracy / R2 | Helped |
| Smooth the training target with a Gaussian kernel (ablation) | Tree-based models and neural networks · Numerical-feature classification benchmark | test accuracy | Mixed |
| Remove low-importance uninformative features (ablation) | MLP-like neural networks · Numerical-feature classification benchmark | test accuracy | Helped |
| Add uninformative Gaussian features (ablation) | MLP-like neural networks · Numerical-feature classification benchmark | test accuracy | Hurt |
| Randomly rotate the feature representation (ablation) | Neural networks, especially Resnet · Numerical-feature classification benchmark | test accuracy | Mixed |
| Remove the least important half of features before rotation (ablation) | Models on numerical tabular data · Numerical-feature classification benchmark | test accuracy | Hurt |
| Use a Random Forest instead of an MLP | Electricity dataset · Electricity | test accuracy: 80% -> 85% | Helped |
Closest papers by meaning
- TabArena: A Living Benchmark for Machine Learning on Tabular Data (NeurIPS 2025 · similarity 0.89)
- GRANDE: Gradient-Based Decision Tree Ensembles for Tabular Data (ICLR 2024 · similarity 0.89)
- Generative Forests (NeurIPS 2024 · similarity 0.88)
- Retrieval & Fine-Tuning for In-Context Tabular Models (NeurIPS 2024 · similarity 0.88)
- TabR: Tabular Deep Learning Meets Nearest Neighbors (ICLR 2024 · similarity 0.88)
- A Data-Centric Perspective on Evaluating Machine Learning Models for Tabular Data (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.