Research map / Probabilistic and scientific modeling / Variational and generative sampling
MissScore: High-Order Score Estimation in the Presence of Missing Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MissScore, a denoising score matching framework that learns first- and second-order (Hessian) scores directly from incomplete data under MCAR/MAR, with variance reduction, and applies them to tabular sampling (Langevin/Ozaki) and causal discovery.
Problem
Learning high-order scores of a data density requires complete data, but real-world data (healthcare, finance) often has missing values; imputation biases results and generative/causal downstream tasks suffer.
Approach
Extends high-order denoising score matching (Meng et al., 2021) with missingness-mask weights (inverse probability weighting under MCAR/MAR) so that the objectives for first- and second-order scores are unbiased with respect to the oracle scores computed from complete data, plus tailored variance-reduction estimators for small noise levels.
Key results
- Variance reduction lowers second-order score MSE at sigma=0.01 from ~15 to ~0.05 on 100-d Gaussian data with no missingness (Table 1).
- On Census with 30% MCAR missingness, Ozaki sampling reaches 88% fidelity and 81% ML efficiency vs 73%/70% for DSM-delete and 82%/77% for STaSy-mean (Table 2).
- Causal discovery performance is comparable to MissDAG and MissOTM while substantially reducing runtime, and outperforms MissDiffAN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MissScore Ozaki rather than DSM-delete | Census data generation · Census, MCAR missing ratio 0.3 | Fidelity: 73% -> 88% | Helped |
| Use MissScore Ozaki rather than STaSy-mean | Census data generation · Census, MCAR missing ratio 0.3 | ML efficiency: 77% -> 81% | Helped |
| Use Ozaki sampling rather than Langevin sampling | MissScore · Census, MCAR missing ratio 0.3 | Fidelity: 86% -> 88% | Helped |
| Add variance reduction to first-order score estimation (ablation) | MissScore s1 · 100-dimensional correlated Gaussian, MCAR, α=0.3, σ=0.01 | MSE: 0.44 ± 0.01 -> 0.15 ± 0.01 | Helped |
| Add variance reduction to second-order score estimation (ablation) | MissScore s2 · 100-dimensional correlated Gaussian, MCAR, α=0.3, σ=0.01 | MSE: 29.74 ± 3.35 -> 0.05 ± 0.00 | Helped |
| Add variance reduction to first-order score estimation (ablation) | MissScore s1 · 100-dimensional correlated Gaussian, MCAR, α=0.3, σ=0.1 | MSE: 0.32 ± 0.01 -> 0.13 ± 0.00 | Helped |
| Add variance reduction to second-order score estimation (ablation) | MissScore s2 · 100-dimensional correlated Gaussian, MCAR, α=0.3, σ=0.1 | MSE: 0.16 ± 0.02 -> 0.04 ± 0.00 | Helped |
| Increase missing ratio (ablation) | MissScore score estimation · 100-dimensional correlated Gaussian, MCAR, σ=0.1 | MSE: s1(VR): 0.07 ± 0.00 -> 0.23 ± 0.00; s2(VR): 0.04 ± 0.00 -> 0.05 ± 0.00 | Hurt |
| Use second-order Ozaki sampling instead of first-order Langevin dynamics | MissScore on Swiss-Roll · Swiss-Roll, MCAR, missing ratio 0.5 | sampling iterations for comparable data quality | Helped |
Surprising findings
- Without variance reduction, second-order score estimation becomes unreliable as sigma approaches zero, with MSE around 15-30 at sigma=0.01.
- The authors note that performance in high-dimensional causal discovery (d=50) is slightly lower than MissDAG/MissOTM.
Limitations
- Accuracy degrades in low-noise settings without variance reduction, especially at high missing ratios.
- Downstream performance (causal discovery, sampling) declines slightly as dimensionality increases.
- Only MCAR/MAR are covered; MNAR is handled heuristically without guarantees.
Experimental setup
Datasets: Bayesian Network (simulated, Ouyang et al., 2023) Census (Adult) synthetic Gaussian synthetic ER-graph causal data
Benchmarks: Census Bayesian Network synthetic causal discovery (ER graphs)
Metrics: MSE of scores SDMetrics fidelity ML efficiency accuracy SHD order divergence runtime
Baselines: DSM-delete STaSy-mean DSM-mean MissDiffAN MissForest MissDAG MissOTM
Builds on: denoising score matching high-order DSM (Meng et al., 2021) Ozaki discretization STaSy DiffAN
Closest papers by meaning
- Score Matching with Missing Data (ICML 2025 · similarity 0.84)
- Generative Modeling under Non-Monotone MAR Missingness via Approximate Wasserstein Gradient Flows (NeurIPS 2026 · similarity 0.82)
- Distribution Shift in Missing Data Imputation: A Risk-Based Perspective and Importance-Weighted Correction under MAR (NeurIPS 2026 · similarity 0.82)
- Learning What Not to Impute: An Uncertainty-Aware Diffusion Framework for Meaningful Missingness (NeurIPS 2026 · similarity 0.82)
- DiffPuter: Empowering Diffusion Models for Missing Data Imputation (ICLR 2025 · similarity 0.82)
- Prediction models that learn to avoid missing values (ICML 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.