Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Sparse Bayesian Generative Modeling for Compressive Sensing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes sparse Bayesian generative modeling (CSVAE and CSGMM) that learns a trainable sparsity-inducing prior in a dictionary's compressible domain from only compressed, noisy observations, and gives a variational lower bound on a sparsity-inducing log-evidence.
Problem
Compressive sensing reconstruction when ground-truth signals are unavailable for training and only compressed, noisy observations can be used to learn a prior.
Approach
Combine sparse Bayesian learning's zero-mean, diagonal Gaussian prior with a latent variable (VAE or GMM) so the conditional Gaussian's variances are learned from data; the model's log-evidence is shown to be upper-bounded by a sparsity-inducing improper prior, so training maximizes a variational lower bound of a sparsity-inducing evidence. Inference needs no optimization loop.
Key results
- On MNIST and piecewise smooth functions, CSVAE and CSGMM achieve the best nMSE and SSIM among trained-from-compressed-data methods, outperforming CSGAN, CKSVD, Lasso and SBL.
- CSVAE and CSGMM can learn effectively from only a few hundred compressed noisy samples (MNIST, M=160).
- CSVAE and CSGMM are considerably faster than the baselines in training and reconstruction time on MNIST (M=200, N_t=20000).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CSVAE or CSGMM trained on compressed observations instead of the compared estimators | MNIST compressive sensing · MNIST | nMSE | Helped |
| Use CSVAE or CSGMM trained on compressed observations instead of the compared estimators | MNIST compressive sensing · MNIST | SSIM | Helped |
| Use CSVAE instead of Lasso, SBL, or CKSVD | Compressive sensing of piecewise smooth functions · Piecewise smooth functions | nMSE | Helped |
| Use CSVAE instead of Lasso, SBL, or CKSVD | Compressive sensing of piecewise smooth functions · Piecewise smooth functions | perception | Helped |
| Use the proposed CSVAE or CSGMM rather than the compared estimators | Compressive sensing of celebA images · celebA | nMSE | Helped |
| Use the proposed CSVAE or CSGMM rather than the compared estimators | Compressive sensing of celebA images · celebA | SSIM | Helped |
| Use different dictionaries rather than the db4 dictionary (ablation) | CSVAE and CSGMM · MNIST | nMSE | No clear effect |
| Apply the models directly in the pixel domain instead of using a fixed dictionary (ablation) | CSVAE and CSGMM · MNIST | nMSE | Hurt |
| Use alternative estimators instead of the conditional mean approximation (ablation) | CSVAE and CSGMM · Compressive sensing experiments | reconstruction performance | No clear effect |
Surprising findings
- Restricting the latent conditional to zero mean introduces a bias, but the learned prior still acts as an effective regularizer for reconstruction, even though it does not capture the true signal distribution.
Limitations
- Zero-mean, diagonal-covariance restriction on the conditional prior introduces a bias that may prevent learning the true distribution of the sparse coefficients.
- Dictionary is fixed; learnable dictionaries are left to future work.
- CSGMM and CSVAE with varying measurement matrices create computational overhead, so CSGMM was not evaluated on piecewise smooth functions.
Experimental setup
Datasets: MNIST celebA FashionMNIST piecewise smooth functions (synthetic 1D HeaviSine-based)
Benchmarks: MNIST celebA piecewise smooth functions FashionMNIST
Metrics: nMSE SSIM training time reconstruction time
Baselines: Lasso SBL CSGAN CKSVD OMP
Builds on: sparse Bayesian learning variational autoencoder generative model-based compressive sensing AmbientGAN
Closest papers by meaning
- Modeling Sparse Deviations for Compressed Sensing using Generative Models (ICML 2018 · similarity 0.90)
- Uncertainty Modeling in Generative Compressed Sensing (ICML 2022 · similarity 0.90)
- Towards Sample-Optimal Compressive Phase Retrieval with Sparse and Generative Priors (NeurIPS 2021 · similarity 0.90)
- How many measurements are enough? Bayesian recovery in inverse problems with general distributions (NeurIPS 2025 · similarity 0.88)
- Preconditioned Langevin Dynamics with Score-based Generative Models for Infinite-Dimensional Linear Bayesian Inverse Problems (NeurIPS 2025 · similarity 0.87)
- Training Bayesian Neural Networks with Sparse Subspace Variational Inference (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.