Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Unrolled denoising networks provably learn to perform optimal Bayesian inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proves that unrolled AMP networks with learned MLP denoisers, trained layerwise with gradient descent on product-prior compressed sensing data, converge to Bayes AMP denoisers, and shows empirical gains over Bayes AMP in low dimensions, non-Gaussian designs, and non-product priors.
Problem
Whether algorithm-unrolled neural networks trained by gradient descent on samples from an unknown prior can provably match the performance of prior-aware Bayes AMP for compressed sensing and rank-one matrix estimation.
Approach
Unroll AMP with learned MLP scalar denoisers trained layerwise; combine state evolution with NTK analysis of one-hidden-layer ReLU networks so that the learning problem reduces to one-dimensional score estimation, giving dimension-free overparametrization guarantees that the learned denoisers approach Bayes-optimal ones.
Key results
- On Bernoulli-Gaussian compressed sensing, LDNet converges to -44.9313 dB NMSE over 15 layers versus -45.3280 dB for Bayes AMP.
- On the Z2 prior with compressed sensing at sigma^2=0.075, LDNet reaches NMSE 0.4267, a 1.28% improvement over Bayes AMP.
- With learnable B at delta=1/2, LDNet outperforms Bayes AMP by 7.28% at m=200, 16.44% at m=150, and 37.06% at m=100 (averaged over five matrices).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use learned denoisers instead of prior-aware Bayes AMP denoisers | LDNet for compressed sensing · Bernoulli-Gaussian prior | NMSE (dB), 15 layers: -44.9313 dB vs -45.3280 dB | No clear effect |
| use learned denoisers instead of prior-aware Bayes AMP denoisers | LDNet for compressed sensing · Z2 prior | NMSE, layer 15: 1.28 % | Helped |
| add trainable B alongside layerwise denoisers | LDNet for compressed sensing · Gaussian sensing matrix, m=200 | NMSE improvement over Bayes AMP: 7.2750 % | Helped |
| add trainable B alongside layerwise denoisers | LDNet for compressed sensing · Gaussian sensing matrix, m=150 | NMSE improvement over Bayes AMP: 16.4364 % | Helped |
| add trainable B alongside layerwise denoisers | LDNet for compressed sensing · Gaussian sensing matrix, m=100 | NMSE improvement over Bayes AMP: 37.0605 % | Helped |
| use learned denoisers instead of prior-aware Bayes AMP denoisers | LDNet for rank-one matrix estimation · Gaussian prior | NMSE | No clear effect |
| use learned denoisers instead of prior-aware Bayes AMP denoisers | LDNet for rank-one matrix estimation · Z2 prior | NMSE | Helped |
| train end-to-end instead of layerwise | LDNet · Compressed sensing | MSE | Hurt |
| initialize each layer's denoiser from the previously learned denoiser | Layerwise-trained LDNet · Compressed sensing | training performance | Helped |
| add auxiliary trainable parameters alongside learned denoisers | LDNet · Non-Gaussian sensing matrices | NMSE | Helped |
Surprising findings
- Learned denoisers transfer to other randomly drawn sensing matrices despite being trained on a fixed matrix.
- Naive end-to-end training gets trapped in suboptimal local assignments of denoisers, motivating layerwise training.
Limitations
- Theory assumes a single coordinate and fresh sensing matrices per sample, a less sample-efficient simplification than experiments.
- Theoretical guarantee restricted to product priors with sub-Gaussian marginals and Lipschitz score; non-product priors lack theory.
Experimental setup
Datasets: synthetic Bernoulli-Gaussian prior samples synthetic Z2 prior samples synthetic Gaussian prior samples synthetic mixture-of-Gaussians prior samples
Benchmarks: synthetic compressed sensing (m=250, d=500) synthetic rank-one matrix estimation (d=1024) low-dimensional non-product prior (d=10)
Metrics: NMSE (dB) NMSE Frobenius MSE
Baselines: Bayes AMP ISTA COSAMP guided denoisers adjusted Bayes AMP
Builds on: approximate message passing learned AMP (LDAMP) neural tangent kernel LISTA
Closest papers by meaning
- Algorithmic Guarantees for Inverse Imaging with Untrained Network Priors (NeurIPS 2019 · similarity 0.88)
- Spectrally Adaptive Channel-aware Unrolling Network for Compressed Sensing (AAAI 2026 · similarity 0.87)
- Proximal Algorithm Unrolling: Flexible and Efficient Reconstruction Networks for Single-Pixel Imaging (CVPR 2025 · similarity 0.87)
- Provable Posterior Sampling with Denoising Oracles via Tilted Transport (NeurIPS 2024 · similarity 0.86)
- How many measurements are enough? Bayesian recovery in inverse problems with general distributions (NeurIPS 2025 · similarity 0.85)
- Time-Embedded Algorithm Unrolling for Computational MRI (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.