Research map / Optimization / Optimizers
On permutation symmetries in Bayesian neural network posteriors: a variational perspective
Simone Rossi, Ankit Singh, Thomas Hannagan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| align the second VI posterior to the first via permutation matching before linear interpolation (ablation) | Gaussian VI BNN (MLP, 3 layers) · MNIST / CIFAR10 | marginalized predictive-likelihood barrier | Improved |
| align the second VI posterior to the first via permutation matching before linear interpolation (ablation) | Gaussian VI BNN (ResNet20) · CIFAR10 | marginalized predictive-likelihood barrier | Improved |
| increase network width with alignment (MLP and ResNet20) | VI with alignment · MLP / ResNet20 on CIFAR10 | likelihood barrier | Improved |
| increase ResNet20 width without alignment | naive VI interpolation · ResNet20 on CIFAR10 | likelihood barrier | No clear effect |
| increase prior variance with alignment | VI with alignment · MLP on MNIST/Fashion-MNIST, ResNet20 x8 on CIFAR10 | likelihood barrier | No clear effect |
| increase prior variance without alignment (more diffuse prior) | naive VI interpolation · MNIST, Fashion-MNIST, CIFAR10 | likelihood barrier | Improved |
| increase prior variance | VI BNN · MNIST, Fashion-MNIST, CIFAR10 | train-test barrier gap | Hurt |
| cold posterior (T<1) versus warm posterior (T>1) with alignment | VI with alignment · MLP/ResNet20 | likelihood barrier | Improved |
| replace VI with SGD MAP plus weight matching, narrow models | ResNet20 · CIFAR10 narrow widths | loss barrier | No clear effect |
| replace VI with SGD MAP plus weight matching, wide models | ResNet20 · CIFAR10 wide widths | loss barrier | Improved |
| replace FRN with LayerNorm (ablation) | BNN ResNet20 with alignment · CIFAR10 | likelihood barrier | Improved |
| train with data augmentation (ablation) | VI with alignment · CIFAR10 | likelihood barrier | No clear effect |
Closest papers by meaning
- A Symmetry-Aware Exploration of Bayesian Neural Network Posteriors (ICLR 2024 · similarity 0.88)
- A Unified Analysis of Stochastic Gradient Descent with Arbitrary Data Permutations and Beyond (NeurIPS 2025 · similarity 0.86)
- A Study of Bayesian Neural Network Surrogates for Bayesian Optimization (ICLR 2024 · similarity 0.86)
- Linear Mode Connectivity between Multiple Models modulo Permutation Symmetries (ICML 2025 · similarity 0.86)
- A Tale of Two Symmetries: Exploring the Loss Landscape of Equivariant Models (NeurIPS 2025 · similarity 0.86)
- Incremental Gradient Descent with Small Epoch Counts is Surprisingly Slow on Ill-Conditioned Problems (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
