Research map / Probabilistic and scientific modeling / Variational and generative sampling
Parameter Estimation in DAGs from Incomplete Data via Optimal Transport
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OTP-DAG, which learns parameters of any directed graphical model with latent variables by minimizing a Wasserstein-based reconstruction objective with backward stochastic maps, avoiding likelihood maximization and posterior inference.
Problem
Parameter estimation in directed graphical models (DAGs) with latent variables from incomplete data, where the marginal likelihood and posterior are intractable without structural or model-class assumptions.
Approach
Recasts parameter learning as optimal transport between observed-data and model distributions; uses local backward stochastic maps from observed nodes to their parents, with a relaxed push-forward constraint penalized by a divergence (Wasserstein in practice), yielding a WAE-like objective that generalizes to any DAG without ELBO or posterior assumptions.
Key results
- On the Poisson HMM, OTP-DAG reached about 3 minutes for 50 steps versus about 20 minutes for EM, with comparable or lower parameter MAE on lambda_2 through lambda_4.
- On topic inference, OTP-DAG achieved the best coherence on all three real datasets (e.g., 10.45% vs 6.71% for Batch EM on 20 News Group).
- On CIFAR10 image reconstruction with K=512 codewords, OTP-DAG improved SSIM from 0.70 to 0.80 and perplexity from 69.8 to 498.6 over VQ-VAE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use OTP-DAG instead of VQ-VAE | Discrete representation learning · CIFAR10 | SSIM: 0.70 -> 0.80 | Helped |
| Use OTP-DAG instead of VQ-VAE | Discrete representation learning · MNIST | PSNR: 33.37 -> 33.62 | Helped |
| Use OTP-DAG instead of VQ-VAE | Discrete representation learning · SVHN | PSNR: 26.94 -> 32.56 | Helped |
| Use OTP-DAG instead of VQ-VAE | Discrete representation learning · CELEBA | PSNR: 27.48 -> 29.77 | Helped |
| Use OTP-DAG instead of Batch EM | LDA topic inference · 20 News Group | Coherence: 6.71 ± 0.16 -> 10.45 ± 0.56 | Helped |
| Use OTP-DAG instead of Batch EM | LDA topic inference · BBC News | Coherence: 8.67 ± 0.62 -> 9.12 ± 0.81 | Helped |
| Use OTP-DAG instead of Batch EM | LDA topic inference · DBLP | Coherence: 4.52 ± 0.53 -> 7.66 ± 0.44 | Helped |
| Use OTP-DAG instead of Prod LDA | LDA topic inference · 20 News Group | Diversity: 92.67 ± 4.51 -> 92.00 ± 2.65 | Hurt |
| Use OTP-DAG instead of SVI | LDA topic inference · BBC News | Diversity: 92.33 ± 2.31 -> 87.67 ± 2.65 | Hurt |
| Use OTP-DAG instead of Prod LDA | LDA topic inference · DBLP | Diversity: 98.67 ± 1.53 -> 97.33 ± 1.53 | Hurt |
| Use OTP-DAG instead of EM | Poisson HMM parameter estimation · Poisson HMM | mean absolute error λ1: 0.022 ± 0.042 -> 0.040 ± 0.129 | Hurt |
| Use OTP-DAG instead of EM | Poisson HMM parameter estimation · Poisson HMM | mean absolute error λ2: 0.088 ± 0.105 -> 0.079 ± 0.088 | Helped |
Surprising findings
- Baselines (EM, SVI) achieve lower KL divergence on the LDA gamma estimates, while OTP-DAG is best by Hellinger and Wasserstein metrics.
- ProdLDA attains higher diversity than OTP-DAG on 20 News Group and DBLP despite lower coherence.
Limitations
- Scaling to larger graphs requires ancestral sampling for model marginals over parents, which is computationally expensive.
- Divergence measure D adds computational cost; GAN-based JS would need extra discriminators, and exact WS is costly in high dimension.
- Reparameterization is required, so discrete or non-reparameterizable distributions need additional approximations such as generalized reparameterization gradients.
Experimental setup
Datasets: 20 News Group BBC News DBLP CIFAR10 MNIST SVHN CelebA synthetic LDA data synthetic Poisson HMM data
Benchmarks: 20 News Group BBC News DBLP CIFAR10 MNIST SVHN CelebA
Metrics: KL divergence Hellinger distance Wasserstein distance coherence (NPMI) diversity mean absolute error runtime SSIM PSNR LPIPS
Baselines: Batch EM SVI ProdLDA VQ-VAE EM
Builds on: Wasserstein auto-encoders variational autoencoders vector quantized VAE Gumbel-softmax
Closest papers by meaning
- Provable Posterior Sampling with Denoising Oracles via Tilted Transport (NeurIPS 2024 · similarity 0.85)
- Generative Modeling under Non-Monotone MAR Missingness via Approximate Wasserstein Gradient Flows (NeurIPS 2026 · similarity 0.85)
- Data Mixture Optimization: A Multi-fidelity Multi-scale Bayesian Framework (NeurIPS 2025 · similarity 0.85)
- Learning Gaussian DAG Models without Condition Number Bounds (ICML 2025 · similarity 0.85)
- CoLiDE: Concomitant Linear DAG Estimation (ICLR 2024 · similarity 0.85)
- Statistical Inference in Causal Partial Identification under Smooth Densities (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.