Research map / Probabilistic and scientific modeling / Bayesian optimization
Thompson Sampling in Function Spaces via Neural Operators
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Neural Operator Thompson Sampling (NOTS), a sample-then-optimize Thompson sampling method that uses retrained neural operators as approximate GP posterior samples to optimize known functionals of an unknown operator's output.
Problem
Optimizing a known functional of an expensive-to-query unknown operator (function-to-function mapping) in function spaces, where queries are costly but functional evaluations are cheap.
Approach
Treat randomly initialized and retrained neural operators (FNO-based) as approximate samples from a vector-valued GP posterior via the infinite-width conjugate kernel, enabling Thompson sampling without explicit operator uncertainty quantification; provides Bayesian regret bounds for the exact GP case.
Key results
- On Darcy flow rate, NOTS final average regret 0.012 ± 0.001 vs best baseline STO-NTS 0.068 and GP-TS 0.674.
- On shallow water inverse problem, NOTS achieves 0.134 ± 0.043 final average regret vs GP-TS 1.942 and BO 1.639.
- On Darcy potential energy, NOTS 0.125 ± 0.042 vs best baseline BFO 0.208 (GP-TS 0.189 with higher variance).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NOTS instead of STO-NTS | Darcy flow optimization · Darcy flow rates | final average regret: 0.068 ± 0.002 -> 0.012 ± 0.001 | Helped |
| Use NOTS instead of GP-TS | Darcy flow optimization · Darcy flow energy | final average regret: 0.189 ± 0.093 -> 0.125 ± 0.042 | Helped |
| Use NOTS instead of GP-TS | Darcy flow optimization · Darcy flow pressure | final average regret: 0.038 ± 0.004 -> 0.012 ± 0.001 | Helped |
| Use NOTS instead of BO | Shallow water inverse problem · Shallow water | final average regret: 1.639 ± 0.532 -> 0.134 ± 0.043 | Helped |
Surprising findings
- STO-NTS performs worse than GP-based BO on Darcy potential energy and shallow water, despite being a neural method.
- Theory requires training only the last layer of a single-hidden-layer model, yet the experiments use multi-layer FNOs trained fully, so the theory does not strictly cover the experiments.
Limitations
- Theory limited to finite search spaces and a single-hidden-layer neural operator with only last layer trained; experiments use multi-layer FNOs.
- Regret bound applies to the exact GP case; the approximate retrained neural operator sample is not formally covered.
- Restricted to 300 iterations and finite domains; batched and continuous-domain extensions left to future work.
Experimental setup
Datasets: Darcy flow (finite-difference solver, 16x16) Shallow water equations on rotating sphere (32x64 grid)
Benchmarks: Darcy flow optimization (flow rates, pressure, potential energy) Shallow water inverse problem
Metrics: cumulative regret final average regret R_T/T
Baselines: Random search BO (log-EI with infinite-width BNN kernel) GP-TS BFO STO-NTS
Builds on: Gaussian process Thompson sampling Neural Thompson Sampling Fourier neural operator neural tangent kernel composite Bayesian optimization
Closest papers by meaning
- Linearization Turns Neural Operators into Function-Valued Gaussian Processes (ICML 2025 · similarity 0.88)
- Bayesian Optimization on Function Spaces via Sparse RKHS Manifolds (NeurIPS 2026 · similarity 0.87)
- Optimizing Posterior Samples for Bayesian Optimization via Rootfinding (ICLR 2025 · similarity 0.87)
- A Study of Bayesian Neural Network Surrogates for Bayesian Optimization (ICLR 2024 · similarity 0.87)
- Sample-efficient Bayesian Optimisation Using Known Invariances (NeurIPS 2024 · similarity 0.86)
- Optimization for Neural Operators can Benefit from Width (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.