Research map / Learning theory and algorithms / Kernels and regression theory
A Bregman Proximal Viewpoint on Neural Operators
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Frames neural operator layers as Bregman proximal minimizers over Banach function spaces, yielding a Bregman Neural Operator (BFNO) with an inverse-activation skip term that trains deeper and more accurately than FNO variants.
Problem
Neural operators built from compositions of kernel layers with pointwise activations are hard to scale in depth, and their design lacks a unifying variational interpretation over function spaces.
Approach
Each operator layer is the minimizer of a Bregman-regularized problem over Lebesgue function spaces, so activations are Bregman proximity operators from dual to primal space. Choosing a Legendre integral functional gives a new Bregman neural operator layer sigma(sigma^{-1}(v) + Kv + b), which reduces to identity when weights vanish.
Key results
- On 2D Navier-Stokes (nu=1e-4), BFNO achieves lower l2 error than FNO for every depth T in {4,8,16,32,64}, while FNO degrades from T=8 onward.
- BFNO keeps improving with depth up to T=64, whereas F-FNO plateaus at T=16.
- Cross-validated comparisons show BFNO consistently superior or comparable across PDE benchmarks, with notable gains on 1D Burgers and 2D Navier-Stokes; 2D Darcy is the exception.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the inverse activation term to FNO layers (BFNO) | FNO · 1D Burgers | test relative error | Helped |
| add the inverse activation term to FNO layers (BFNO) | FNO · 2D Navier-Stokes (ν = 10⁻⁴) | test relative error | Helped |
| add the inverse activation term to FNO layers (BFNO) | FNO · 2D Darcy | test relative error | Hurt |
| increase depth from T=4 to T=64 | BFNO · 2D Navier-Stokes (ν = 10⁻⁴) | prediction error | Helped |
| increase depth beyond T=8 | FNO · 2D Navier-Stokes (ν = 10⁻⁴) | prediction error | Hurt |
| increase depth beyond T=16 | ResFNO · 2D Navier-Stokes (ν = 10⁻⁴) | prediction error | Hurt |
| increase depth beyond T=16 | F-FNO · 2D Navier-Stokes (ν = 10⁻⁴) | prediction error | No clear effect |
| add batch normalization layers (ablation) | BFNO · PDE benchmarks | test error | No clear effect |
| replace SoftPlus with ReLU (ablation) | BFNO · PDE benchmarks | test error | No clear effect |
| use BFNO instead of FNO | Fourier operator models · Low-frequency bands across PDE datasets | relative error | Helped |
| use BFNO instead of FNO | Fourier operator models · High-frequency bands across PDE datasets | relative error | Mixed |
Surprising findings
- On 2D Darcy, F-FNO outperforms BFNO, which the authors attribute to F-FNO's extra MLP layers.
Limitations
- Universal approximation is proven only for sigmoidal-type activations, while experiments use ReLU-like activations (SoftPlus with large beta), a gap the authors acknowledge.
- On high-frequency error bands and on complex datasets, improvements over FNO are marginal.
- Monotone activation operators only; non-monotone activations such as GeLU and swish are not covered.
Experimental setup
Datasets: 2D Navier-Stokes (nu=1e-4) 1D Burgers 2D Darcy
Benchmarks: FNO PDE benchmarks PDEBench
Metrics: l2 relative error
Baselines: FNO F-FNO ResFNO WNO
Builds on: Fourier Neural Operator Bregman proximity operator PDEBench
Closest papers by meaning
- MgNO: Efficient Parameterization of Linear Operators via Multigrid (ICLR 2024 · similarity 0.87)
- Optimization for Neural Operators can Benefit from Width (ICML 2025 · similarity 0.87)
- Beyond Regular Grids: Fourier-Based Neural Operators on Arbitrary Domains (ICML 2024 · similarity 0.86)
- Optimal deep learning of holomorphic operators between Banach spaces (NeurIPS 2024 · similarity 0.86)
- Amortized Fourier Neural Operators (NeurIPS 2024 · similarity 0.84)
- Can neural operators always be continuously discretized? (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.