Research map / Learning theory and algorithms / Kernels and regression theory
Optimal deep learning of holomorphic operators between Banach spaces
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that standard feedforward DNNs trained with l2 loss, paired with approximate encoders/decoders, achieve near-optimal generalization rates for learning holomorphic operators between Banach spaces, with matching lower bounds up to log terms.
Problem
Learning holomorphic operators between Banach spaces from finite noisy data, with generalization bounds in L2 and L-infinity norms that are optimal in the number of training samples m.
Approach
Combines arbitrary approximate encoders/decoders with problem-agnostic DNN families (width and depth depending only on m, width exceeding depth) trained by l2-loss minimization, proving algebraic generalization rates for Banach-valued holomorphic operators, that uncountably many fully-connected minimizers achieve these rates, and that no learning procedure can beat them up to logs.
Key results
- Best-performing DNNs on the parametric diffusion problem roughly match the predicted rate m^-1, consistent with the theory's effective rate m^(1/2-1/p) for p<2/3.
- ELU and tanh architectures generally outperform ReLU architectures across the three PDE experiments.
- Lower bounds show the rate m^(1/2-1/p) in L2 (and m^(1-1/p) in L-infinity for i.i.d. sampling) cannot be beaten by any procedure, up to log factors.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ELU or tanh instead of ReLU activation | DNNs for parametric elliptic diffusion · Parametric elliptic diffusion | relative L² error | Helped |
| Increase parametric dimension from 4 to 8 | DNNs for parametric elliptic diffusion · Parametric elliptic diffusion | relative L² error | No clear effect |
| Use ELU or tanh instead of ReLU activation | DNNs for Navier-Stokes-Brinkman velocity · Parametric Navier-Stokes-Brinkman | relative L² error | Helped |
| Increase parametric dimension from 4 to 8 | DNNs for Navier-Stokes-Brinkman velocity · Parametric Navier-Stokes-Brinkman | relative L² error | No clear effect |
| Use ELU or a smaller tanh network | DNNs for stationary Boussinesq · Parametric stationary Boussinesq | relative L² error | Helped |
| Increase parametric dimension | DNNs for stationary Boussinesq · Parametric stationary Boussinesq | relative L² error | No clear effect |
| Increase training sample count | Best-performing DNN architectures across the PDE experiments · Parametric elliptic diffusion, Navier-Stokes-Brinkman, and stationary Boussinesq | relative L² error: m^{-1} | Helped |
Surprising findings
- Adaptive sampling (active learning) gives no benefit for the optimal rate, since i.i.d. sampling already achieves it up to log terms.
Limitations
- Theory guarantees only that some minimizers are good, not all minimizers, and does not yet cover standard training dynamics.
- Banach-valued output bounds are weaker by a factor 1/2 in the approximation error than the Hilbert-valued case.
- Cannot yet handle Gaussian input measures.
Experimental setup
Datasets: synthetic FEM-generated parametric PDE solutions (elliptic diffusion, Navier-Stokes-Brinkman, Boussinesq)
Benchmarks: parametric elliptic diffusion parametric Navier-Stokes-Brinkman parametric stationary Boussinesq
Metrics: relative L2 Bochner error
Baselines: ReLU DNNs
Builds on: PCA-Net DeepONet
Closest papers by meaning
- MgNO: Efficient Parameterization of Linear Operators via Multigrid (ICLR 2024 · similarity 0.89)
- Optimization for Neural Operators can Benefit from Width (ICML 2025 · similarity 0.88)
- Neural Operators with Localized Integral and Differential Kernels (ICML 2024 · similarity 0.88)
- In-Context Multi-Operator Learning with DeepOSets (NeurIPS 2026 · similarity 0.88)
- Data-Efficient Operator Learning via Unsupervised Pretraining and In-Context Learning (NeurIPS 2024 · similarity 0.88)
- Learning semilinear neural operators: A unified recursive framework for prediction and data assimilation. (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.