Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Efficient Low Rank Gaussian Variational Inference for Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase rank K from 1 to 5 (ablation) | ELRG-VI · Vectorized MNIST MLP 400 units | test error rate (%): 1.82 ± 0.25% -> 1.54 ± 0.18% | Helped |
| Use ELRG-VI K=5 instead of MF-VI | MLP · Vectorized MNIST 400 units | test error rate (%): 2.51 ± 0.09% -> 1.54 ± 0.18% | Helped |
| Use proposed reparametrization instead of naive low-rank sampling (ablation) | ELRG-VI · Vectorized MNIST 400 units | test error rate (%): 3.00 ± 1.82% -> 1.82 ± 0.25% (K=1) | Helped |
| Share one variational sample across inputs in a batch (ablation) | ELRG-VI · Vectorized MNIST | validation log-likelihood: -0.057 -> -0.112 | Hurt |
| Add low-rank terms K>0 to the diagonal ELRG-D-VI posterior (ablation) | ELRG-D-VI · Vectorized MNIST 2x100 MLP | negative log-likelihood (underfitting) | No clear effect |
| Add low-rank constant-diagonal covariance (K>0) to MF-VI | ELRG-VI · Vectorized MNIST 2x100 MLP | held-out log-likelihood: -0.076 -> -0.057 | Helped |
| Use ELRG-VI K=5 instead of MF-VI | LeNet · MNIST | negative test log-likelihood: 0.0462 ± 0.002 -> 0.0257 ± 0.002 | Helped |
| Use ELRG-VI K=5 instead of MF-VI | LeNet · CIFAR10 | test error rate (%): 28.89 ± 0.16% -> 29.43 ± 0.32% | Hurt |
| Use ELRG-VI K=4 instead of MAP | ResNet18 · CIFAR10 | test ECE: 0.08 ± 0.003 -> 0.018 ± 0.002 | Helped |
| Use ELRG-VI K=4 instead of MAP | ResNet18 · CIFAR10 | test accuracy (%): 87.66 ± 0.18% -> 87.24 ± 0.37% | Hurt |
| Use ELRG-VI K=4 instead of MC Dropout | AlexNet · CIFAR10 | test log-likelihood: -0.717 ± 0.01 -> -0.678 ± 0.006 | Helped |
| Increase K from 1 to 4 (ablation) | ELRG-VI · CIFAR100 ResNet18 | test ECE: 0.154 ± 0.005 -> 0.096 ± 0.005 | Helped |
Closest papers by meaning
- VIKING: Deep variational inference with stochastic projections (NeurIPS 2025 · similarity 0.91)
- Variational Inference with Mixtures of Isotropic Gaussians (NeurIPS 2025 · similarity 0.90)
- Provably Scalable Black-Box Variational Inference with Structured Variational Families (ICML 2024 · similarity 0.89)
- Training Bayesian Neural Networks with Sparse Subspace Variational Inference (ICLR 2024 · similarity 0.89)
- Bayesian Low-Rank Learning (Bella): A Practical Approach to Bayesian Neural Networks (AAAI 2025 · similarity 0.88)
- Nearly Dimension-Independent Convergence of Mean-Field Black-Box Variational Inference (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.