Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Neural Spectral Methods: Self-supervised learning in the spectral domain
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Neural Spectral Methods, which learn PDE solution operators in an orthogonal-basis spectral domain and train with a spectral loss (Parseval's identity) instead of a PINN quadrature loss, without solution data.
Problem
Learning parametric PDE solution operators without solution data while avoiding the optimization difficulties, aliasing, and high cost of grid-based PINN losses with neural operators such as FNO.
Approach
Represent the solution and network layers in an orthogonal (Fourier/Chebyshev) basis so that the PDE residual's norm is computed exactly from spectral coefficients via Parseval's identity, yielding a resolution-independent spectral neural operator trained with a self-supervised spectral loss.
Key results
- On periodic Poisson, NSM reaches 0.057% L2 relative error versus 1.75% for FNO x256^2 + PINN.
- On reaction-diffusion, NSM achieves roughly 0.08% error across diffusion coefficients, while grid-based FNO+PINN and CNO+PINN reach 0.2-13% error; NSM matches a numerical solver at about 10x speedup.
- On unforced Navier-Stokes, NSM trains in 10h versus 41h to 131h for FNO+PINN and reaches 0.71% to 3.53% error versus 8-18% for FNO+PINN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use spectral-domain NSM instead of FNO with PINN loss | PDE solution learning · Periodic Poisson | L2 relative error (%): 4.24 ± 0.13 -> .057 ± .012 | Helped |
| use spectral-domain NSM instead of FNO with PINN loss | PDE solution learning · Dirichlet Poisson | L2 relative error (%): 3.52 ± 0.29 -> 0.280 ± 0.017 | Helped |
| train the NSM base architecture with PINN loss instead of spectral loss (ablation) | NSM/CNO · Reaction-Diffusion | L2 relative error (%): 0.20 ± 0.01 -> .075 ± .016; 0.48 ± 0.16 -> .086 ± .019; 0.78 ± 0.01 -> .083 ± .006; 1.28 ± 0.42 -> .077 ± .005 | Hurt |
| increase the PINN training grid resolution (ablation) | FNO · Reaction-Diffusion | L2 relative error (%): 2.30 ± 0.19 -> 0.33 ± 0.04 (ν=0.005); 2.64 ± 0.97 -> 0.57 ± 0.19 (ν=0.01) | Helped |
| train with spectral loss instead of PINN loss (ablation) | NSM · Reaction-Diffusion (ν=0.01) | training cost (GFLOP): 200.3 / 25.0 -> 15.0 / 0.32 | Helped |
| use NSM instead of FNO with PINN loss | PDE solution learning · Unforced Navier-Stokes | L2 relative error (%): 8.18 ± 2.83 -> 0.71 ± 0.02 (ν=10⁻²); 14.81 ± 0.67 -> 1.65 ± 0.26 (ν=10⁻³); 17.88 ± 2.67 -> 3.53 ± 0.53 (ν=10⁻⁴) | Helped |
| use NSM instead of FNO with PINN loss | PDE solution learning · Long transient forced Navier-Stokes (ν=1/500) | L2 relative error (%): 55.1 ± 17.4 -> 13.2 ± 0.57 | Helped |
| replace NSM with T1 architecture trained using spectral loss (ablation) | PDE solution learning · Unforced Navier-Stokes (ν=10⁻⁴) | L2 relative error (%): 3.53 ± 0.53 -> 9.22 ± 0.05 | Hurt |
| train on a fixed-resolution grid and interpolate at inference instead of spectral training (ablation) | FNO · Reaction-Diffusion | L2 relative error (%): 55.7 ± 0.01 (ν=0.005); 76.9 ± 0.02 (ν=0.01); 79.5 ± 0.04 (ν=0.05); 80.4 ± 0.23 (ν=0.1) | Hurt |
| apply activations at collocation points rather than directly to spectral coefficients (ablation) | spectral neural operator · Reaction-Diffusion as diffusivity increases | solution robustness | Helped |
| increase the PINN training grid resolution (ablation) | FNO · Periodic Poisson | L2 relative error (%): 4.24 ± 0.13 -> 2.01 ± 0.03 -> 1.75 ± 0.02 | Helped |
| increase diffusion coefficient | NSM and grid-based methods · Reaction-Diffusion | L2 relative error (%): NSM: .075 ± .016, .086 ± .019, .083 ± .006, .077 ± .005; FNO×256²+PINN: 0.33 ± 0.04, 0.57 ± 0.19, 1.06 ± 0.28, 4.07 ± 2. | Mixed |
Surprising findings
- FNO with PINN loss collapses during training with ReLU and needed tanh activations to train at all.
- Fixed-grid training of FNO followed by interpolation fails to converge, with errors above 55%.
Limitations
- Efficient spectral representation requires sufficiently smooth solutions; singularities need many more basis functions.
- Designed for low-dimensional problems; extension to high-dimensional problems left for future work.
Experimental setup
Datasets: online-sampled random fields for PDE parameters
Benchmarks: 2D Poisson equation 1D reaction-diffusion equation 2D Navier-Stokes (unforced and forced)
Metrics: L2 relative error PDE residual error wall-clock time GFLOP
Baselines: FNO + PINN loss SNO + spectral loss TransformOnce (T1) CNO + PINN loss standard operator splitting numerical solver
Builds on: Fourier Neural Operator physics-informed neural networks Spectral Neural Operator TransformOnce Galerkin projection
Closest papers by meaning
- Solving High-Dimensional PDEs with Latent Spectral Models (ICML 2023 · similarity 0.91)
- Neuro-Spectral Architectures for Causal Physics-Informed Networks (NeurIPS 2025 · similarity 0.89)
- Spectral-Refiner: Accurate Fine-Tuning of Spatiotemporal Fourier Neural Operator for Turbulent Flows (ICLR 2025 · similarity 0.88)
- Beyond Regular Grids: Fourier-Based Neural Operators on Arbitrary Domains (ICML 2024 · similarity 0.88)
- Solving Differential Equations with Constrained Learning (ICLR 2025 · similarity 0.87)
- Hyperbolic-PDE GNN: Spectral Graph Neural Networks in the Perspective of A System of Hyperbolic Partial Differential Equations (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.