Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Linearization Turns Neural Operators into Function-Valued Gaussian Processes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LUNO, which linearizes a trained neural operator (Fourier neural operator case study) around Gaussian weight-space beliefs to obtain a function-valued Gaussian process over the operator's outputs, via a probabilistic version of currying.
Problem
Neural operators give point predictions for PDE solution operators without reliable predictive uncertainty; quantifying it is needed for high-stakes simulation use.
Approach
Push a Gaussian weight-space posterior (e.g., linearized Laplace) through the linearized neural operator and use probabilistic currying to obtain a Banach-space-valued (function-valued) Gaussian process over the operator, which is post-hoc, resolution-agnostic and cheap to evaluate at arbitrary output points.</221>
Key results
- On Burgers' equation with 25 trajectories, LUNO-LA reaches the best NLL (-2.0787) versus -1.1572 for Sample-LA, with RMSE 3.62e-02 comparable to the ensemble's 3.49e-02.
- On the Pos-Neg-Flip OOD set, LUNO-LA achieves expected marginal NLL 1.164 versus 27.046 for Sample-LA and 494.935 for input perturbations, while deep ensembles achieve -1.014.
- Deep ensembles obtain the lowest NLL in next-step prediction on the OOD sets, but their empirical covariance is rank-deficient, whereas LUNO-LA covariance rank is bounded only by the number of parameters.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LUNO-LA instead of Sample-LA | FNO uncertainty estimation · Burgers' equation, 25 trajectories | RMSE: 5.59e-02 -> 3.62e-02 | Helped |
| Use LUNO-LA instead of Sample-LA | FNO uncertainty estimation · Burgers' equation, 25 trajectories | NLL: -1.1572 -> -2.0787 | Helped |
| Use LUNO-Iso instead of Sample-Iso | FNO uncertainty estimation · Burgers' equation, 25 trajectories | NLL: -1.9341 -> -1.9488 | Helped |
| Use LUNO-LA instead of Sample-LA | FNO uncertainty estimation · Advection-Diffusion OOD, Base | NLL: -2.576 -> -2.934 | Helped |
| Use LUNO-LA instead of Sample-LA | FNO uncertainty estimation · Advection-Diffusion OOD, Flip | NLL: 4.395 -> -1.126 | Helped |
| Use LUNO-LA instead of Sample-LA | FNO uncertainty estimation · Advection-Diffusion OOD, Pos-Neg-Flip | NLL: 27.046 -> 1.164 | Helped |
| Use LUNO-Iso instead of Sample-Iso | FNO uncertainty estimation · Advection-Diffusion OOD, Base | NLL: -2.921 -> -2.892 | Hurt |
| Use LUNO-Iso instead of Sample-Iso | FNO uncertainty estimation · Advection-Diffusion OOD, Flip | NLL: 4.071 -> 3.450 | Helped |
| Use LUNO-Iso instead of Sample-Iso | FNO uncertainty estimation · Advection-Diffusion OOD, Pos-Neg-Flip | NLL: 43.362 -> 37.733 | Helped |
| Use LUNO-LA instead of Ensemble | FNO uncertainty estimation · Advection-Diffusion OOD, Flip | NLL: -3.825 -> -1.126 | Hurt |
Surprising findings
- Deep ensembles achieve the lowest expected marginal NLL in several OOD next-step settings despite LUNO-LA's advantages in covariance structure.
- Sample-based approximations of the function-valued posterior perform substantially worse in NLL than their linearized counterparts.
Limitations
- Modeling weight-space covariances is challenging; the main limitation noted by the authors.
- Case study restricted to last Fourier block of FNO for the efficient representation.
- Numerical instabilities and near-singular Jacobians can reduce the covariance rank in practice.
Experimental setup
Datasets: Burgers' equation 2D Advection-Diffusion equation
Benchmarks: Burgers' equation (25 trajectories) Advection-Diffusion OOD (Flip, Pos, Neg, Pos-Neg-Flip)
Metrics: RMSE expected marginal chi^2 expected marginal NLL
Baselines: Input perturbations Deep ensemble Sample-Iso Sample-LA
Builds on: Fourier neural operator linearized Laplace approximation Gaussian processes
Closest papers by meaning
- Thompson Sampling in Function Spaces via Neural Operators (NeurIPS 2025 · similarity 0.88)
- Light-Weight Diffusion Multiplier and Uncertainty Quantification for Fourier Neural Operators (NeurIPS 2025 · similarity 0.87)
- MgNO: Efficient Parameterization of Linear Operators via Multigrid (ICLR 2024 · similarity 0.87)
- Guaranteed Approximation Bounds for Mixed-Precision Neural Operators (ICLR 2024 · similarity 0.87)
- Gradients of Functions of Large Matrices (NeurIPS 2024 · similarity 0.87)
- Using Uncertainty Quantification to Characterize and Improve Out-of-Domain Learning for PDEs (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.