Research map / Image generation / Fast diffusion sampling
Score Matching with Missing Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adapts score matching (classical, truncated, sliced, denoising) to partially missing data via marginal score objectives, with an importance-weighting estimator (with finite-sample bounds) and a variational gradient estimator, evaluated on parameter estimation and Gaussian graphical model edge detection.
Problem
Score matching assumes fully observed data, but many real datasets are incomplete; existing missing-data score methods are mostly tied to neural diffusion models or lack explicit parametric support.
Approach
Replace the score with marginal scores over observed coordinates and estimate them either by importance-weighted Monte Carlo (with finite-sample bounds for truncated score matching) or by a gradient-first variational approach that lets the inner expectations be estimated by any black-box method, applicable to any parametric score model.
Key results
- On the truncated Gaussian experiment, Marg-IW and EM achieve the lowest Fisher divergence while naive Zeroed Score Matching does not converge.
- On the non-Gaussian model, Marg-Var yields the lowest error and its advantage over other methods widens as dimension increases.
- For Gaussian graphical model edge detection on S&P 100 and yeast data, Marg-Var clearly outperforms the other methods, which perform comparably with each other.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Marg-IW or EM instead of Marg-Var or Zeroed Score Matching | Truncated Gaussian model · 10-dimensional truncated Gaussian parameter estimation | Fisher divergence | Helped |
| Use Marg-IW or EM instead of Zeroed Score Matching | Truncated Gaussian model · 10-dimensional truncated Gaussian parameter estimation | Fisher divergence | Helped |
| Use Marg-Var instead of the competing methods | Non-Gaussian model · ICA-inspired parameter estimation | Fisher divergence | Helped |
| Increase dimensionality while using Marg-Var rather than competing methods | Non-Gaussian model · ICA-inspired parameter estimation | Fisher divergence | Helped |
| Replace Marg-IW with EM | Non-Gaussian model · ICA-inspired parameter estimation | Fisher divergence | No clear effect |
| Use Marg-Var instead of the other approaches | Gaussian graphical model estimation · Star-shaped truncated graph | AUC | Helped |
| Use Marg-Var instead of the other approaches | Gaussian graphical model estimation · Unstructured dense graph | AUC | Helped |
| Increase the number of star centres | Gaussian graphical model estimation · Graphs with increasing number of star centres | AUC | No clear effect |
| Use Marg-Var instead of the other approaches | Gaussian graphical model estimation · S&P 100 stock data | AUC | Helped |
| Use Marg-Var instead of the other approaches | Gaussian graphical model estimation · Yeast gene expression data | AUC | Helped |
| Use Marg-IW rather than Marg-Var in lower-dimensional settings with less data | Missing score matching · Lower-dimensional parameter estimation with less data | Fisher divergence | Helped |
Surprising findings
- The EM approach and Marg-IW give indistinguishable results across experiments, despite being derived differently; the paper leaves the mechanism unexplained.
- Naive zeroing-out of missing coordinates does not converge even in the truncated Gaussian setting, and is only moderately competitive on unstructured graphs.
Limitations
- Finite-sample bounds are proven only for truncated IW score matching on bounded domains and are measured with respect to the marginal Fisher divergence.
- Importance weighting approach struggles in higher dimensions and to reach sqrt(n) rates needs the number of importance samples to grow with n, giving O(n^2) cost.
- The MNAR extension assumes the missingness probabilities are known.
Experimental setup
Datasets: S&P 100 stock closing prices yeast genetic expression data (262 segregants, 7086 genes subset to 106) simulated Gaussian and non-Gaussian data
Benchmarks: simulated Fisher divergence (truncated Gaussian, non-Gaussian model) graph edge detection AUC on simulated GGMs S&P 100 GGM recovery yeast GGM recovery
Metrics: Fisher divergence AUC mean estimation error
Baselines: EM score matching (fractional importance weighting) Zeroed Score Matching MissDiff
Builds on: score matching truncated score matching sliced score matching denoising score matching MissDiff
Closest papers by meaning
- Training Unbiased Diffusion Models From Biased Dataset (ICLR 2024 · similarity 0.86)
- Diffusion Models and the Manifold Hypothesis: Log-Domain Smoothing is Geometry Adaptive (NeurIPS 2025 · similarity 0.85)
- Dimension-free Score Matching and Time Bootstrapping for Diffusion Models (NeurIPS 2025 · similarity 0.85)
- Diffusion Transformers for Imputation: Statistical Efficiency and Uncertainty Quantification (NeurIPS 2025 · similarity 0.85)
- Neural Network-Based Score Estimation in Diffusion Models: Optimization and Generalization (ICLR 2024 · similarity 0.85)
- Identification and Estimation for Nonignorable Missing Data: A Data Fusion Approach (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.