AI Paper Atlas

Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes

Scale-Invariant Empirical-Bayes Laplace Approximation for ReLU Networks

Shivam Pal, Piyush Rai

NeurIPS 2026 · Probabilistic and scientific modeling › Uncertainty and Gaussian processes · approach: Nonparametric and PAC-Bayes Bayesian learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a scale-invariant empirical-Bayes Laplace approximation for ReLU networks that removes dependence on the representative of the positive-homogeneity scale orbit.

Problem

Standard empirical-Bayes Laplace approximation for ReLU networks is not invariant to function-preserving rescaling of parameters, because positive homogeneity creates continuous scale orbits of equivalent parameters while the isotropic objective depends on the chosen representative through prior nor

Approach

Makes the post-hoc Bayesian evidence and uncertainty estimates invariant to function-preserving reparameterizations by accounting for the continuous scale orbits induced by ReLU positive homogeneity, rather than using the standard isotropic prior and objective.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Apply gamma-canonicalization before EB Laplacediag-GGN EB-Laplace · Fashion-MNIST MLP orbit stress teststd of log lambda*: 0.356 -> 0Improved
Apply gamma-canonicalization before EB Laplacediag-GGN EB-Laplace · Fashion-MNIST MLP orbit stress testNLL: 0.3906 -> 0.3644Improved
Apply gamma-canonicalization before EB Laplacediag-GGN EB-Laplace · Fashion-MNIST MLP orbit stress testECE: 0.0682 -> 0.0389Improved
Vary gamma from 0 to 1 in canonicalization (ablation)gamma-canonicalized EB · Fashion-MNIST MLP orbit stress testECE: 0.0387 -> 0.0562Mixed
Apply gamma-canonicalization before KFAC-LaplaceKFAC-Laplace · CIFAR-10 ConvNetNLL: 0.877 -> 0.734Improved
Apply gamma-canonicalization before KFAC-LaplaceKFAC-Laplace · CIFAR-10 ConvNetECE: 0.248 -> 0.130Improved
Apply gamma-canonicalization before KFAC-LaplaceKFAC-Laplace · CIFAR-10 ConvNetaccuracy: 0.774 -> 0.773No clear effect
Apply gamma-canonicalization before last-layer EB-Laplace on rescaled ResNet-18last-layer EB-Laplace · CIFAR-100 ResNet-18NLL: 1.1478 -> 0.7912Improved
Apply gamma-canonicalization to the original (well-scaled) ResNet representativelast-layer EB-Laplace · CIFAR-100 ResNet-18NLL: 0.7906 -> 0.7912No clear effect
Rescale the width-64 MLP by sigma=1.0 before standard EBstandard EB model selection · CIFAR-10 MLP width 64 vs 128ranking reversal rate: 0.0% -> 100.0%Hurt

Experimental setup

Baselines: standard empirical-Bayes Laplace approximation

Builds on: Laplace approximation empirical Bayes

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy