Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes

Uncertainty Quantification with the Empirical Neural Tangent Kernel

NeurIPS 2025 · Probabilistic and scientific modeling › Uncertainty and Gaussian processes · approach: Single-model predictive uncertainty · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes NUQLS, a post-hoc sampling UQ method that runs (stochastic) gradient descent on linearized trained networks from random perturbations to build an ensemble that approximates an NTK Gaussian process posterior.

Problem

Quantifying predictive uncertainty of overparameterized neural networks cheaply and reliably; existing Bayesian UQ methods are either cheap or reliable but not both, and deep ensembles are expensive.

Approach

Linearize a trained network around its parameters and run (S)GD from zero-mean isotropic Gaussian perturbations projected onto the Jacobian null space; the resulting interpolating ensemble samples from a noise-free GP posterior with the empirical NTK, under convexity assumptions on the loss.

Key results

• On UCI regression, NUQLS matches or beats DE, LLA and SWAG on NLL and ECE and is the fastest method, e.g. Energy 8.4 s vs 102.2 s for DE.
• On image classification, NUQLS outperforms all compared methods including deep ensembles on VMSP-based separation of ID-correct, ID-incorrect and OoD points (qualitative/median and skew measures).
• Empirically the NUQLS predictive variance converges toward the NTK-GP variance as epochs and ensemble members increase (SEV metric).

Measured effects reported in the paper

Surprising findings

• Post-hoc LLA-style methods often perform poorly relative to deep ensembles, which are frequently excluded from BDL comparisons.
• Bayesian methods are argued to be unnecessary for improving prediction accuracy; the paper focuses UQ evaluation on variance over softmax outputs.

Limitations

• Theoretical GP connection does not extend to cross-entropy loss, which violates the convexity assumptions.
• Performance degrades when the target network is poorly trained and the linearization is inaccurate.
• Requires full row-rank Jacobian and interpolation assumptions for the theory to hold.

Experimental setup

Datasets: UCI regression datasets (Energy, Kin8nm, Protein, Song) toy cubic regression data FashionMNIST CIFAR-10 CIFAR-100 MNIST SVHN ImageNet

Benchmarks: UCI regression toy regression FashionMNIST/CIFAR-10/CIFAR-100 VMSP uncertainty SVHN/ImageNet ResNet50

Metrics: RMSE NLL ECE wall-clock time VMSP median and skew SEV

Baselines: Deep Ensembles SWAG Linearized Laplace Approximation MC-Dropout Laplace Approximation Variational Inference BatchEnsemble SNGP Bayesian Deep Ensembles Sketched Lanczos Uncertainty

Builds on: Linearized Laplace Approximation Deep Ensembles Neural Tangent Kernel Sampling-LLA

Closest papers by meaning

• Epistemic Uncertainty Quantification For Pre-Trained Neural Networks (CVPR 2024 · similarity 0.91)
• Non-Asymptotic Uncertainty Quantification in High-Dimensional Learning (NeurIPS 2024 · similarity 0.89)
• Linear Uncertainty Quantification of Graphical Model Inference (NeurIPS 2024 · similarity 0.88)
• Efficient Analytic Uncertainty Quantification for Multimodal Regression (NeurIPS 2026 · similarity 0.88)
• Uncertainty Quantification for Deep Regression using Contextualised Normalizing Flows (NeurIPS 2025 · similarity 0.88)
• Uncertainty Quantification for Forward and Inverse Problems of PDEs via Latent Global Evolution (AAAI 2024 · similarity 0.88)