Research map / Probabilistic and scientific modeling / Variational and generative sampling
Multilayer Matrix Factorization via Dimension-Reducing Diffusion Variational Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes dimension-reducing diffusion variational inference (DRD-VI) for multilayer matrix factorization, embedding dimension reduction inside a variational diffusion process, and evaluates it on hyperspectral unmixing and low-dimensional representation clustering.
Problem
Maximum-likelihood inference for multilayer matrix factorization (MMF) requires a good variational distribution; existing VI for MMF uses VAEs, and standard diffusion VI assumes latent dimensions equal data dimensions, so it cannot directly handle MMF's unequal layer dimensions.
Approach
A variational diffusion process whose layers gradually reduce dimension via semi-orthogonal maps U_t and noise, with each MMF layer matched to a variational layer through a light-weight one-layer nonlinear decoder, yielding a per-layer tractable ELBO with closed-form layer terms.
Key results
- On hyperspectral abundance estimation, DRD-VI achieves the lowest MSE on SAMSON (0.328), JASPER (0.305), and URBAN (0.677) among compared methods, and consistently outperforms VASCA.
- On APEX, MiSiCNet (0.413) beats DRD-VI (0.609), so DRD-VI is not uniformly best.
- On low-dimensional clustering, DRD-VI is described as generally performing well and comparable to or sometimes outperforming other MMF methods across six datasets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DRD-VI instead of VASCA | Hyperspectral abundance estimation · SAMSON | MSE: 0.401 -> 0.328 | Helped |
| Use DRD-VI instead of PRISM | Hyperspectral abundance estimation · JASPER | MSE: 0.452 -> 0.305 | Helped |
| Use DRD-VI instead of MISICNET | Hyperspectral abundance estimation · APEX | MSE: 0.413 -> 0.609 | Hurt |
| Use DRD-VI instead of CNNAEU | Hyperspectral abundance estimation · URBAN | MSE: 0.700 -> 0.677 | Helped |
Surprising findings
- On the APEX dataset, the deep-structure baseline MiSiCNet outperforms DRD-VI in MSE by a wide margin.
Limitations
- Experimental results are on relatively small hyperspectral and clustering benchmarks with 10 random initializations.
- Requires a tractable variational latent distribution (Dirichlet, Beta, Gaussian) with analytic mean, covariance and entropy, and a relaxed bound for the final layer.
- Performance is mixed; DRD-VI loses to MiSiCNet on APEX.
Experimental setup
Datasets: SAMSON JASPER APEX URBAN CMU PIE Caltech 101 Silhouettes Fashion MNIST GTSRB DTD Oxford-IIIT Pet
Benchmarks: SAMSON JASPER APEX URBAN CMU PIE Caltech 101 Silhouettes Fashion MNIST GTSRB DTD Oxford-IIIT Pet
Metrics: MSE ARI accuracy NMI
Baselines: SISAL PRISM CNNAEU MiSiCNet VASCA SNMF LC-DMF DMF Deep Semi-NMF DANMF
Builds on: variational diffusion models variational autoencoder hierarchical VAE PRISM
Closest papers by meaning
- Deep Attentive Variational Inference (ICLR 2022 · similarity 0.88)
- Gaussian Process Modeling of Approximate Inference Errors for Variational Autoencoders (CVPR 2022 · similarity 0.88)
- Variational Laplace Autoencoders (ICML 2019 · similarity 0.88)
- DiffEnc: Variational Diffusion with a Learned Encoder (ICLR 2024 · similarity 0.87)
- Generalization in VAE and Diffusion Models: A Unified Information-Theoretic Analysis (ICLR 2025 · similarity 0.87)
- Denoising Diffusion Variational Inference: Diffusion Models as Expressive Variational Posteriors (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.