Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Amortized Fourier Neural Operators
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AM-FNO, which replaces FNO's per-frequency kernel parameters with a fixed-size neural parameterization (KAN or Chebyshev-embedded MLP) so it can use all frequency modes with few parameters.
Problem
Fourier Neural Operators need separate parameters per retained frequency mode, causing parameter growth with PDE dimensionality and forcing frequency truncation that loses high-frequency detail.
Approach
Use a neural network (KAN, or MLP with orthogonal Chebyshev embedding of frequencies, factorized across dimensions) to define the Fourier-space kernel as a function of frequency, so arbitrarily many modes are covered by a fixed parameter budget and no truncation is needed.
Key results
- AM-FNO (KAN) and AM-FNO (MLP) reduce relative error on average by 22% and 31% versus the top baseline across six benchmarks.
- On CFD-2D, AM-FNO (MLP) reaches test error 2.16e-3 versus 4.52e-3 for U-FNO, the best baseline.
- Zero-shot super-resolution on NS-2D trained at 32x32: AM-FNO (MLP) gets 8.73e-2 at 64x64 versus 1.28e-1 for FNO trained at that resolution.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AM-FNO (KAN) instead of the strongest reported baseline | PDE benchmarks · Darcy; NS-2D; Pipe; Airfoil; Elasticity; CFD-1D; CFD-2D | l2 relative error: Darcy: 7.01e-3 -> 4.28e-3; NS-2D: 1.22e-1 -> 1.08e-1; Pipe: 5.20e-3 -> 3.54e-3; Airfoil: 6.39e-3 -> 6.06e-3; Elasticity: | Helped |
| Use AM-FNO (MLP) instead of the strongest reported baseline | PDE benchmarks · Darcy; NS-2D; Pipe; Airfoil; Elasticity; CFD-1D; CFD-2D | l2 relative error: Darcy: 7.01e-3 -> 4.21e-3; NS-2D: 1.22e-1 -> 8.51e-2; Pipe: 5.20e-3 -> 3.44e-3; Airfoil: 6.39e-3 -> 5.64e-3; Elasticity: | Helped |
| Add orthogonal embedding to the MLP parameterization (ablation) | AM-FNO (MLP) · Darcy; Airfoil; Pipe | l2 relative error: Darcy: 1.29e-2 -> 4.21e-3; Airfoil: 7.21e-3 -> 5.64e-3; Pipe: 7.84e-3 -> 3.44e-3 | Helped |
| Replace Chebyshev basis functions with triangular basis functions (ablation) | AM-FNO (MLP) · Darcy; Airfoil; Pipe | l2 relative error: Darcy: 4.21e-3 -> 4.13e-3; Airfoil: 5.64e-3 -> 7.60e-3; Pipe: 3.44e-3 -> 3.80e-3 | Mixed |
| Replace Chebyshev basis functions with non-orthogonal polynomial basis functions (ablation) | AM-FNO (MLP) · Darcy; Airfoil; Pipe | l2 relative error: Darcy: 4.21e-3 -> 1.77e-2; Airfoil: 5.64e-3 -> 1.30e-2; Pipe: 3.44e-3 -> 1.03e-2 | Hurt |
| Use the vanilla FNO architecture (ablation) | AM-FNO (MLP) · Darcy; Airfoil; Pipe | l2 relative error: Darcy: 4.21e-3 -> 5.25e-3; Airfoil: 5.64e-3 -> 8.03e-3; Pipe: 3.44e-3 -> 3.89e-3 | Hurt |
| Remove dimensional factorization (ablation) | AM-FNO (MLP) · Darcy; Airfoil; Pipe | l2 relative error: Darcy: 4.21e-3 -> 5.24e-3; Airfoil: 5.64e-3 -> 5.89e-3; Pipe: 3.44e-3 -> 3.83e-3 | Mixed |
| Increase the number of orthogonal basis functions (ablation) | AM-FNO (MLP) · Darcy; Airfoil | l2 relative error | Helped |
| Increase KAN hidden layer size (ablation) | AM-FNO (KAN) · Darcy; Airfoil | l2 relative error | Helped |
| Increase KAN spline grid size (ablation) | AM-FNO (KAN) · Darcy; Airfoil | l2 relative error | Helped |
| Train on 32 × 32 resolution and evaluate at 64 × 64 | AM-FNO (MLP) · NS-2D, 64 × 64 | l2 relative error: 1.28e-1 -> 8.73e-2 | Helped |
Surprising findings
- Removing the orthogonal embedding from the MLP variant substantially degrades accuracy, and the KAN variant is slower and more memory-hungry than the MLP variant.
- Naively keeping all frequency modes in FNO (FNO+) overfits, with lower train error but higher test error than FNO.
Limitations
- Benchmarks are idealized physical systems and exclude real-world complex problems.
- Extremely high-dimensional PDEs remain challenging due to FFT complexity.
- Each experiment was run only once, so no error bars are reported.
Experimental setup
Datasets: Darcy NS-2D Pipe Airfoil Elasticity CFD-1D CFD-2D
Benchmarks: Darcy NS-2D Pipe Airfoil Elasticity CFD-1D CFD-2D
Metrics: l2 relative error fL2 frequency-band error GPU memory training time per epoch
Baselines: FNO Geo-FNO U-FNO F-FNO AFNO OFormer LSM
Builds on: Fourier Neural Operator Adaptive Fourier Neural Operator F-FNO Kolmogorov-Arnold Networks
Closest papers by meaning
- Maximum Update Parametrization and Zero-Shot Hyperparameter Transfer for Fourier Neural Operators (ICML 2025 · similarity 0.91)
- Guaranteed Approximation Bounds for Mixed-Precision Neural Operators (ICLR 2024 · similarity 0.90)
- Beyond Regular Grids: Fourier-Based Neural Operators on Arbitrary Domains (ICML 2024 · similarity 0.90)
- Component Fourier Neural Operator for Singularly Perturbed Differential Equations (AAAI 2024 · similarity 0.89)
- Improved Operator Learning by Orthogonal Attention (ICML 2024 · similarity 0.89)
- Neural Operators with Localized Integral and Differential Kernels (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.