Research map / Learning theory and algorithms / Kernels and regression theory
Can neural operators always be continuously discretized?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows via category theory that diffeomorphisms between Hilbert spaces admit no continuous finite-dimensional diffeomorphic approximations, and that strongly monotone neural operator layers (and bilipschitz operators decomposed into them plus an isometry) can be continuously discretized.
Problem
Whether neural operators (bijective, i.e. diffeomorphic, between Hilbert spaces) can be discretized continuously by finite-dimensional diffeomorphisms, so that discretization is consistent as dimension grows.
Approach
Frames discretization as a functor from a category of Hilbert-space diffeomorphisms to finite-dimensional approximation sequences; proves a no-go theorem for general diffeomorphisms via orientation/degree arguments, then shows strongly monotone neural operator layers admit continuous linear-projection discretizations, and that bilipschitz operators decompose into strongly monotone layers plus a re
Key results
- No functor from Hilbert-space diffeomorphisms to finite-dimensional diffeomorphic approximations can be both continuous and satisfy the approximation property (A) (no-go theorem).
- Strongly monotone neural operator layers admit continuous linear-projection discretizations A_lin(F)=P_V F|_V satisfying approximation property (A).
- Bilipschitz neural operator layers can be written as a composition of strongly monotone layers H_J...H_1 composed with a reflection A_0 on any bounded ball, with J=O(epsilon^-2) for C^2 operators.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Approximate general Hilbert-space diffeomorphisms by finite-dimensional diffeomorphisms using a continuous approximation functor | General Hilbert-space diffeomorphisms · Hilbert spaces and finite-dimensional subspaces | continuous discretization | Hurt |
| Restrict the class to strongly monotone diffeomorphisms and use linear projection discretization | Strongly monotone neural operator layers · Finite-dimensional subspaces of Hilbert spaces | continuous discretization | Helped |
| Represent a bilipschitz layer as strongly monotone layers plus an identity or reflection | Bilipschitz neural operator layers · Any bounded ball in a Hilbert space | strongly monotone layer decomposition | Helped |
| Use residual finite-rank neural operators with strongly monotone blocks and an identity or reflection | Bilipschitz neural operator layers · Closed ball B_X(0,R) | uniform approximation error: ≤ ε | Helped |
| Use ReCU residual blocks for the stated approximation construction | Finite-rank residual neural operators · Bilipschitz neural operator layers | local invertibility | Helped |
Surprising findings
- Even nonlinear discretizations cannot continuously approximate general infinite-dimensional diffeomorphisms, because the orientation-preserving and orientation-reversing components of diffeomorphisms connect in infinite dimensions.
- Linear projection discretization is continuous only in the weak sense for general strongly monotone diffeomorphisms, but is fully continuous for strongly monotone neural operator layers.
Limitations
- The approximation of bilipschitz operators relies on a fixed reflection and the number of layers J depends on the ball radius R.
- The no-go theorem is stated for norm topology with specific approximation conditions; the positive results require strong monotonicity assumptions that are restrictive.
- Theoretical paper with no experiments on practical neural operator training.
Experimental setup
Builds on: neural operators Fourier neural operator DeepONet
Closest papers by meaning
- Quantitative Approximation for Neural Operators in Nonlinear Parabolic Equations (ICLR 2025 · similarity 0.86)
- Optimal deep learning of holomorphic operators between Banach spaces (NeurIPS 2024 · similarity 0.85)
- MgNO: Efficient Parameterization of Linear Operators via Multigrid (ICLR 2024 · similarity 0.85)
- Beyond Regular Grids: Fourier-Based Neural Operators on Arbitrary Domains (ICML 2024 · similarity 0.84)
- A Bregman Proximal Viewpoint on Neural Operators (ICML 2025 · similarity 0.84)
- Optimization for Neural Operators can Benefit from Width (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.