Research map / Image generation / Image restoration
A Data-Driven Prism: Multi-View Source Separation with Diffusion Model Priors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DDPRISM, an EM framework that learns independent diffusion-model priors for each source from multi-view, noisy, incomplete linear observations, using joint posterior sampling via MMPS.
Problem
Multi-view source separation: recovering unknown source priors and posteriors from linear mixtures of sources observed under multiple views with known mixing matrices, noise, and incomplete or variable-resolution data, without assuming source models or contrastive datasets.
Approach
Embed independent diffusion model priors for each source in a Monte Carlo EM loop; the E-step samples the joint posterior over all sources using a moment-matching (MMPS-style) likelihood score across views, and the M-step retrains each source's diffusion denoiser on its posterior samples. Handles non-contrastive, mixed, and incomplete settings.
Key results
- On GMNIST full-resolution, DDPRISM-Joint reaches posterior FID 1.57 and PSNR 25.60 vs. CLVM-VAE FID 18.87 and PSNR 14.56.
- On 1D manifold contrastive 2-source, DDPRISM-Joint posterior PSNR 38.27 vs. best baseline CLVM-VAE 17.15.
- On downsampled GMNIST, DDPRISM-Joint posterior FID 2.36 vs. best baseline PCPCA 121.7.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DDPRISM-Joint instead of the strongest listed baseline, CLVM - VAE | GMNIST full-resolution · GMNIST full-resolution | FID (posterior): 18.87 -> 1.57 | Helped |
| use DDPRISM-Joint instead of the strongest listed baseline, PCPCA | GMNIST downsampled · GMNIST downsampled | FID (posterior): 121.7 -> 2.36 | Helped |
| replace the default U-Net denoiser with an MLP (ablation) | DDPRISM-Joint · GMNIST full-resolution | FID (posterior): 1.57 -> 49.03 | Hurt |
| use a small U-Net instead of the default U-Net (ablation) | DDPRISM-Joint · GMNIST full-resolution | FID (posterior): 1.57 -> 2.47 | Hurt |
| train for only 2 EM laps instead of 64 (ablation) | DDPRISM-Joint · GMNIST full-resolution | FID (posterior): 1.57 -> 96.85 | Hurt |
| train for 8 EM laps instead of 64 (ablation) | DDPRISM-Joint · GMNIST full-resolution | FID (posterior): 1.57 -> 0.04 | Helped |
| train for 32 EM laps instead of 64 (ablation) | DDPRISM-Joint · GMNIST full-resolution | FID (posterior): 1.57 -> 2.26 | Hurt |
| remove Gaussian-prior initialization (ablation) | DDPRISM-Joint · GMNIST full-resolution | FID (posterior): 1.57 -> 10.41 | Hurt |
| use 1/4th, 1/16th, or 1/64th of the dataset (ablation) | DDPRISM-Joint · GMNIST full-resolution | FID (posterior): 1.57 -> 4.64; 10.19; 38.34 | Hurt |
| use only 16 PC sampling steps (ablation) | DDPRISM-Joint · GMNIST full-resolution | FID (posterior): 1.57 -> 5.41 | Hurt |
| replace the MLP encoder with a full-depth U-Net (ablation) | CLVM-VAE · GMNIST full-resolution | FID: 18.87 -> 20.14 | Hurt |
| use full-depth U-Nets for both encoder and decoder (ablation) | CLVM-VAE · GMNIST full-resolution | FID: 18.87 -> 388.26 | Hurt |
Surprising findings
- The mixed 1D manifold problem remains identifiable with mixing matrices that vary between sources even when the fraction f_mix=1.0, while fixed-mixing f_mix=1.0 is provably non-identifiable.
- Gibbs sampling converged much more slowly than joint sampling even with eight times more compute per EM lap.
Limitations
- Restricted to linear mixing; cannot model nonlinear processes like occlusion.
- Requires exact knowledge of mixing matrices and Gaussian noise assumption.
- Expensive sampling and EM training limit resolution, dataset size, and number of sources; performance degrades with smaller datasets.
Experimental setup
Datasets: Grassy MNIST (ImageNet grass crops + MNIST 0/1 digits) synthetic 1D manifolds in R^5 COSMOS HST archival observations Galaxy Zoo Hubble catalog
Benchmarks: 1D manifold contrastive MVSS 1D manifold mixed MVSS Grassy MNIST full-resolution Grassy MNIST downsampled COSMOS galaxy images
Metrics: Sinkhorn divergence PQMass p-value PSNR FID standard deviation
Baselines: PCPCA CLVM-Linear CLVM-VAE DDPRISM-Gibbs
Builds on: MMPS Monte Carlo EM (rozet2024learning) Gibbs diffusion (heurtel_depeiges2024listening) EDM (karras2022elucidating) CLVM
Closest papers by meaning
- Unified Framework for Diffusion Generative Models in SO(3): Applications in Computer Vision and Astrophysics (AAAI 2024 · similarity 0.85)
- Listening to the noise: Blind Denoising with Gibbs Diffusion (ICML 2024 · similarity 0.84)
- Sourcerer: Sample-based Maximum Entropy Source Distribution Estimation (NeurIPS 2024 · similarity 0.84)
- A Mixture-Based Framework for Guiding Diffusion Models (ICML 2025 · similarity 0.84)
- Analyzing and Guiding Zero-Shot Posterior Sampling in Diffusion Models (NeurIPS 2026 · similarity 0.84)
- Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Data (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.