Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Posterior Contraction for Sparse Neural Networks in Besov Spaces with Intrinsic Dimensionality
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows sparse Bayesian neural networks with spike-and-slab or continuous shrinkage priors attain near-minimax posterior contraction rates over anisotropic and composite Besov spaces, with rate adaptation via depth/width/sparsity priors.
Problem
Posterior contraction and rate adaptation of Bayesian neural networks for regression over anisotropic Besov spaces and their compositions, where existing Bayesian theory covers only isotropic or simpler classes and anisotropic results rely on fractional posteriors.
Approach
Places sparse spike-and-slab or continuous shrinkage priors on ReLU network weights (with depth, width, sparsity priors for adaptation) and shows the standard posterior contracts at n^{-s~/(2s~+1)}(log n)^{3/2}, where s~ is the intrinsic smoothness, extending to composite anisotropic Besov classes.
Key results
- Posterior contracts at rate n^{-s~/(2s~+1)}(log n)^{3/2} for sparse spike-and-slab and shrinkage priors over anisotropic Besov classes, matching minimax rate up to log factors.
- Rate adaptation is achieved by placing priors on depth, width and sparsity, so the rate holds without knowledge of smoothness.
- Same near-minimax rate extends to composite anisotropic Besov classes, with intrinsic smoothness s~* replacing s~.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Place priors on network width D and sparsity S instead of setting network parameters using the unknown Besov smoothness | Spike-and-slab BNN · Anisotropic Besov spaces | posterior contraction rate: n^{-\tilde{s}/(2\tilde{s}+1)}(\log n)^{3/2} | Helped |
| Use a polynomial-tailed slab instead of an exponential-tailed slab | Spike-and-slab BNN · Anisotropic Besov spaces | posterior contraction rate: n^{-\tilde{s}/(2\tilde{s} + 1)} (\log n)^2 | Hurt |
Limitations
- Horseshoe and other widely used shrinkage priors are not covered by the assumptions.
- Shrinkage prior assumptions require mixture weight near 1 on a near-zero component, which is practically undesirable.
- Theory is for fully connected ReLU networks and Gaussian regression, with no experiments.
Experimental setup
Metrics: posterior contraction rate
Baselines: Suzuki 2019 frequentist DNN Besov Kong et al. Bayesian Besov spike-and-slab fractional posterior anisotropic Besov BNN (Bayesian dense) Polson and Ročková spike-and-slab BNN Hölder
Builds on: Besov spaces posterior contraction theory (Ghosal and van der Vaart) sparse ReLU network approximation theory
Closest papers by meaning
- Posterior Contraction Rates for sparse Kolmogorov-Arnold Networks in Anisotropic Besov Spaces (NeurIPS 2026 · similarity 0.94)
- Training Bayesian Neural Networks with Sparse Subspace Variational Inference (ICLR 2024 · similarity 0.89)
- An Iterative Min-Min Optimization Method for Sparse Bayesian Learning (ICML 2024 · similarity 0.87)
- A Study of Bayesian Neural Network Surrogates for Bayesian Optimization (ICLR 2024 · similarity 0.86)
- Bayesian Optimization via Continual Variational Last Layer Training (ICLR 2025 · similarity 0.85)
- High-Probability Minimax Adaptive Estimation in Besov Spaces via Online-to-Batch (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.