Research map / Probabilistic and scientific modeling / Variational and generative sampling
Latent variable model for high-dimensional point process with structured missingness
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LLSM and LLPPSM, GP-prior deep latent variable models (VAEs) with separate latent variables for observations and missingness masks, optionally coupled with a GP-based Hawkes-style temporal point process, for longitudinal high-dimensional data with structured missingness.
Problem
Modeling high-dimensional longitudinal data that has structured (MAR/MNAR) missingness, correlated observations over time, and irregular measurement times generated by an unknown stochastic point process.
Approach
Combine GP-prior VAEs with a separate latent variable and GP prior for the missingness mask (MNAR via dependence on both latents), a longitudinal additive kernel, and a GP point process whose intensity is fed into the kernels of observation and mask latents, trained with scalable amortised variational inference using KL upper bounds and mini-batching.
Key results
- On regularly sampled Health MNIST, LLSM achieves imputation MSE 0.144 vs 0.283 for L-VAE at 75% max missingness and 0.174 vs 0.373 at 90%.
- On irregularly sampled Health MNIST, LLPPSM gives the best 50% and 75% imputation MSE (0.128 and 0.162) and best future data prediction at all levels (e.g., 0.025 at 50%).
- On Physionet future prediction, LLSM achieves MSE 0.713 versus 0.720 for L-VAE, while LLPPSM is worse at 0.745.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LLSM instead of L-VAE | Health MNIST regular · Health MNIST regular, 75% and 90% missing | imputation MSE: 0.283 \sd{\pm 0.062} -> 0.144 \sd{\pm 0.009}; 0.373 \sd{\pm 0.001} -> 0.174 \sd{\pm 0.016} | Helped |
| Use LLSM instead of L-VAE | Health MNIST regular · Health MNIST regular, 50% missing | future data prediction MSE: 0.021 \sd{\pm 0.002} -> 0.023 \sd{\pm 0.002} | Hurt |
| Use LLSM instead of L-VAE | Health MNIST regular · Health MNIST regular, 75% and 90% missing | future data prediction MSE: 0.038 \sd{\pm 0.008} -> 0.024 \sd{\pm 0.001}; 0.047 \sd{\pm 0.000} -> 0.026 \sd{\pm 0.002} | Helped |
| Use LLSM instead of L-VAE | Health MNIST regular · Health MNIST regular, 50%, 75%, and 90% missing | future missingness prediction MSE: 0.032 \sd{\pm 0.000} -> 0.031 \sd{\pm 0.002}; 0.038 \sd{\pm 0.002} -> 0.038 \sd{\pm 0.003}; 0.040 \sd{\pm 0.002} -> 0.03 | No clear effect |
| Add a temporal point process to LLSM | Irregular Health MNIST · Health MNIST irregular, 50% and 75% missing | imputation MSE: 0.130 \sd{\pm 0.007} -> 0.128 \sd{\pm 0.005}; 0.163 \sd{\pm 0.011} -> 0.162 \sd{\pm 0.007} | Helped |
| Add a temporal point process to LLSM | Irregular Health MNIST · Health MNIST irregular, 90% missing | imputation MSE: 0.191 \sd{\pm 0.012} -> 0.207 \sd{\pm 0.028} | Hurt |
| Add a temporal point process to LLSM | Irregular Health MNIST · Health MNIST irregular, future data prediction | future data prediction MSE: 0.030 \sd{\pm 0.001} -> 0.025 \sd{\pm 0.002}; 0.031 \sd{\pm 0.001} -> 0.027 \sd{\pm 0.001}; 0.031 \sd{\pm 0.001} -> 0.03 | Helped |
| Add a temporal point process to LLSM | Irregular Health MNIST · Health MNIST irregular, future missingness prediction | future missingness prediction MSE: 0.063 \sd{\pm 0.002} -> 0.054 \sd{\pm 0.006}; 0.067 \sd{\pm 0.001} -> 0.056 \sd{\pm 0.005}; 0.066 \sd{\pm 0.001} -> 0.05 | Helped |
| Add a temporal point process to LLSM | Physionet · Physionet | future prediction MSE: 0.713 \sd{\pm 0.006} -> 0.745 \sd{\pm 0.009} | Hurt |
Surprising findings
- On Physionet, the point-process variant (LLPPSM) performs worse than LLSM, which the authors attribute to regularly hourly-sampled measurements making the temporal process pseudo-stochastic and redundant.
- On regular Health MNIST at 50% missingness, L-VAE slightly beats LLSM on future data prediction (0.021 vs 0.023).
Limitations
- Physionet data lacks ground truth for missing values, so imputation could not be measured there
- Point-process component adds complexity and did not help on the regularly sampled Physionet data
- Experiments rely on simulated Health MNIST with MSE as the only metric
Experimental setup
Datasets: Health MNIST Physionet 2012 Challenge
Benchmarks: Health MNIST (regular) Health MNIST (irregular) Physionet 2012
Metrics: MSE
Baselines: mean imputation GPPVAE GRUI-GAN L-VAE
Builds on: GP-prior VAE L-VAE GPPVAE GRUI-GAN Titsias sparse variational GP
Closest papers by meaning
- Preventing Model Collapse in Gaussian Process Latent Variable Models (ICML 2024 · similarity 0.86)
- Amortized Structured Stochastic Variational Inference for Gaussian Process Latent Variable Models (NeurIPS 2026 · similarity 0.86)
- Neighbour-Driven Gaussian Process Variational Autoencoders for Scalable Structured Latent Modelling (ICML 2025 · similarity 0.86)
- Identifying Metric Structures of Deep Latent Variable Models (ICML 2025 · similarity 0.86)
- Bayesian Basis Function Approximation for Scalable Gaussian Process Priors in Deep Generative Models (ICML 2025 · similarity 0.85)
- Multi-modal Gaussian Process Variational Autoencoders for Neural and Behavioral Data (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.