Research map / Learning theory and algorithms / Prediction and decision losses
Imputation for prediction: beware of diminishing returns.
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Empirical study on 19 tabular regression datasets relating imputation accuracy to downstream prediction accuracy across four imputers, three predictors (MLP, SAINT, XGBoost), with and without missingness indicators, finding imputation gains translate into only small prediction gains.
Problem
Whether investing in more advanced imputation methods yields meaningfully better predictions for models trained on data with missing values, given theory and prior benchmarks suggesting simple constant imputation can be consistent and competitive.
Approach
Quantifies, per dataset and model, the slope and correlation between imputation R² and prediction R² across controlled MCAR and MNAR settings, and studies how the missingness indicator, model expressiveness, and response linearity modulate this relationship.
Key results
- Gains in prediction R² are typically about 10% or less of gains in imputation R²; for XGBoost average slope is around 0.025 or less.
- Correlation between imputation and prediction accuracy is systematically lower for real outcomes than for linear outcomes, by roughly 0.1 to 0.3 across models.
- Adding the missingness indicator benefits prediction even under MCAR, and reduces the effect size of imputation accuracy on prediction.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use more accurate imputation | Prediction models across real-data datasets · MCAR | prediction R^2: an improvement of 0.1 in imputation R^2 typically leads to an improvement in prediction R^2 that is 10 times smaller | Helped |
| Use more accurate imputation | XGBoost · Real-data datasets, MCAR | prediction R^2: 0.025 or less | No clear effect |
| Use more accurate imputation | MLP rather than SAINT or XGBoost · Real-data datasets, MCAR, without missingness indicator | prediction R^2 | Helped |
| Add the missingness indicator (ablation) | Prediction models · MCAR | prediction performance | Helped |
| Add the missingness indicator (ablation) | Prediction models · MCAR | effect of imputation accuracy on prediction performance | Hurt |
| Use more accurate imputation | Prediction models with missingness indicator · MCAR | prediction performance | No clear effect |
| Use more accurate imputation | XGBoost and SAINT · MNAR self-censoring | prediction accuracy | Hurt |
| Add a shuffled missingness indicator instead of the true indicator (ablation) | Prediction models · Real outcomes, 50% MCAR | prediction performance | Hurt |
| Add a shuffled missingness indicator instead of the true indicator (ablation) | XGBoost · Real outcomes, 50% MCAR | prediction performance | No clear effect |
| Add a shuffled missingness indicator (ablation) | Prediction models · Real outcomes, 50% MCAR | effect of imputation accuracy on prediction accuracy | No clear effect |
| Add the true missingness indicator rather than a shuffled indicator (ablation) | Prediction models · MCAR | prediction performance | Helped |
| Increase the missing rate (ablation) | Prediction models · MCAR | imputation-accuracy effect slope | Helped |
Surprising findings
- The missingness indicator improves prediction even under MCAR, where it carries no information about the outcome.
- Under MNAR self-censoring without the indicator, improving imputation has on average a negative effect on XGBoost and SAINT predictions.
Limitations
- Focuses on numerical features and regression tasks; categorical features are excluded by design.
- MCAR treated as best case; MNAR results limited to one self-censoring mechanism (probit).
- Imputation R² is an imperfect proxy for downstream prediction gains.
Experimental setup
Datasets: 19 OpenML tabular regression datasets (OpenML benchmark 336) semi-synthetic linear-outcome versions of the 19 datasets
Benchmarks: 19 OpenML tabular regression datasets
Metrics: R^2 (test) imputation R^2 regression slope partial correlation critical difference (Nemenyi)
Baselines: mean imputation iterativeBR missforest condexp XGBoost native missing handling SAINT native missing handling
Builds on: MissForest MIA (missing incorporated in attribute) SAINT XGBoost Optuna
Closest papers by meaning
- Random features models: a way to study the success of naive imputation (ICML 2024 · similarity 0.88)
- Distribution Shift in Missing Data Imputation: A Risk-Based Perspective and Importance-Weighted Correction under MAR (NeurIPS 2026 · similarity 0.87)
- Learning What Not to Impute: An Uncertainty-Aware Diffusion Framework for Meaningful Missingness (NeurIPS 2026 · similarity 0.85)
- Prediction models that learn to avoid missing values (ICML 2025 · similarity 0.85)
- Diffusion Transformers for Imputation: Statistical Efficiency and Uncertainty Quantification (NeurIPS 2025 · similarity 0.85)
- Missing data and cluster graphs: cluster-level missingness vs variable-level missingness (NeurIPS 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.