Research map / Learning theory and algorithms / Prediction and decision losses
Fast, Accurate, and Simple Models for Tabular Data via Augmented Distillation
Rasool Fakoor, Jonas W Mueller, Nick Erickson, Pratik Chaudhari, Alexander J Smola
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Gibbs-sampled augmented data using 1 Gibbs round for distillation | Selected student model · Regression problems | explained variation: 88.63 -> 89.35 | Improved |
| add Gibbs-sampled augmented data using 1 Gibbs round for distillation | Selected student model · Binary classification | accuracy: 82.85 -> 84.21 | Improved |
| add Gibbs-sampled augmented data using 1 Gibbs round for distillation | Selected student model · Multiclass classification | accuracy: 80.80 -> 82.88 | Improved |
| use 5 Gibbs rounds instead of 1 (ablation) | FAST-DAD distillation · Regression problems | explained variation: 89.35 -> 89.25 | Mixed |
| use 5 Gibbs rounds instead of 1 (ablation) | FAST-DAD distillation · Binary classification | accuracy: 84.21 -> 84.04 | Mixed |
| use 5 Gibbs rounds instead of 1 (ablation) | FAST-DAD distillation · Multiclass classification | accuracy: 82.88 -> 82.76 | Mixed |
| use 10 Gibbs rounds instead of 1 (ablation) | FAST-DAD distillation · Regression problems | explained variation: 89.35 -> 89.09 | Hurt |
| use 10 Gibbs rounds instead of 1 (ablation) | FAST-DAD distillation · Binary classification | accuracy: 84.21 -> 83.74 | Hurt |
| use 10 Gibbs rounds instead of 1 (ablation) | FAST-DAD distillation · Multiclass classification | accuracy: 82.88 -> 82.64 | Hurt |
| replace teacher probability targets with hard class predictions during MUNGE augmentation | MUNGE student distillation · Multiclass classification | accuracy | Hurt |
Closest papers by meaning
- Hybrid Autoencoders for Tabular Data: Leveraging Model-Based Augmentation in Low-Label Settings (NeurIPS 2025 · similarity 0.87)
- TabArena: A Living Benchmark for Machine Learning on Tabular Data (NeurIPS 2025 · similarity 0.86)
- TabEBM: A Tabular Data Augmentation Method with Distinct Class-Specific Energy-Based Models (NeurIPS 2024 · similarity 0.86)
- GRANDE: Gradient-Based Decision Tree Ensembles for Tabular Data (ICLR 2024 · similarity 0.86)
- TabM: Advancing tabular deep learning with parameter-efficient ensembling (ICLR 2025 · similarity 0.86)
- A Data-Centric Perspective on Evaluating Machine Learning Models for Tabular Data (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
