Research map / Learning theory and algorithms / Prediction and decision losses
Prediction models that learn to avoid missing values
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes missingness-avoiding (MA) learning, adding a missingness-reliance penalty to decision trees, LASSO, random forests and gradient-boosted trees so that predictions rarely need missing features, with comparable AUROC to unregularized baselines.
Problem
Models deployed with missing test-time feature values either need imputation (bias, reduced interpretability) or missingness indicators (added complexity), so predictions depend on missing values and become hard to interpret.
Approach
Regularize learning objectives with a penalty on the frequency with which a model's decision path or nonzero coefficients use features that are missing for the sample, so models learn to avoid missing features while keeping accuracy; tree splits are penalized contextually, while LASSO uses feature-specific penalties proportional to missingness rate.
Key results
- MA-DT reduces missingness reliance on ADNI from 11.7% (DT) to 0.1% with AUROC 74.1 vs 72.7 for DT.
- MA-LASSO vs LR reduces average missingness reliance by 66.0 points with confidence intervals for AUROC overlapping in all cases except linear models on FICO.
- On NHANES, LR and RF have missingness reliance of 100% while MA-LASSO and MA-RF reduce it to 0.4% and 0.2%, with comparable AUROC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add missingness-reliance regularization to decision trees | DT · ADNI, FICO, LIFE, NHANES | missingness reliance: average decrease of 4.4 | Helped |
| Add missingness-rate-weighted sparsity regularization | LASSO · ADNI, FICO, LIFE, NHANES | missingness reliance: average decrease of 66.0 | Helped |
| Add missingness-reliance regularization to random forests | RF · ADNI, FICO, LIFE, NHANES | missingness reliance: average decrease of 70.2 | Helped |
| Add missingness-reliance regularization to gradient-boosted trees | XGBoost · ADNI, FICO, LIFE, NHANES | missingness reliance: reduces missingness reliance by 64.4 on average | Helped |
| Add missingness-reliance regularization to gradient-boosted trees | XGBoost · ADNI, FICO, LIFE, NHANES | AUROC | Hurt |
| Set the regularization parameter to α=0 instead of α=∞ (ablation) | MA models · ADNI | AUROC | Helped |
| Set the regularization parameter to α=0 instead of α=∞ (ablation) | MA models · ADNI | missingness reliance | Hurt |
| Increase tree maximum depth as an alternative to missingness regularization (ablation) | MA-DT · LIFE | AUROC | Helped |
| Apply missingness regularization with α≤1.0 (ablation) | MA-DT · LIFE | missingness reliance: approximately 30% to approximately 10% | Helped |
| Apply missingness regularization with α≤1.0 (ablation) | MA-DT · LIFE | AUROC | No clear effect |
| Introduce MNAR rather than MAR synthetic missingness (ablation) | MA-LASSO · Breast Cancer | AUROC | Hurt |
| Introduce synthetic missingness on an increasing proportion of features (ablation) | MA-LASSO · Breast Cancer | AUROC | Hurt |
Surprising findings
- Standard decision trees and LR can have very high missingness reliance on structured-missingness data (e.g., NHANES), where redundant features are used despite being frequently missing.
- Reducing max tree depth is a less effective way than the alpha penalty to control missingness reliance, since it hurts AUROC.
Limitations
- Penalty may underperform when missingness is informative (MNAR or label-dependent), since missingness indicators are not used.
- Not robust to distribution shift in missingness patterns; evaluated only on in-distribution test sets.
- Missingness reliance is a binary max-based metric; avoiding missing values entirely may not be feasible.
Experimental setup
Datasets: NHANES LIFE ADNI Breast Cancer Pharyngitis FICO
Benchmarks: ADNI FICO LIFE NHANES Breast Cancer Pharyngitis
Metrics: AUROC missingness reliance (rho-hat)
Baselines: LR DT RF M-GAM XGBoost NeuMiss MINTY
Builds on: CART decision trees Lasso XGBoost MINTY M-GAM
Closest papers by meaning
- Interpretable Generalized Additive Models for Datasets with Missing Values (NeurIPS 2024 · similarity 0.87)
- Learning What Not to Impute: An Uncertainty-Aware Diffusion Framework for Meaningful Missingness (NeurIPS 2026 · similarity 0.87)
- Random features models: a way to study the success of naive imputation (ICML 2024 · similarity 0.86)
- Distribution Shift in Missing Data Imputation: A Risk-Based Perspective and Importance-Weighted Correction under MAR (NeurIPS 2026 · similarity 0.85)
- Imputation for prediction: beware of diminishing returns. (ICLR 2025 · similarity 0.85)
- Leveraging Predictive Equivalence in Decision Trees (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.