Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Spectral-Refiner: Accurate Fine-Tuning of Spatiotemporal Fourier Neural Operator for Turbulent Flows
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ST-FNO, a spatiotemporal adaptation of Fourier neural operators mapping between Bochner spaces, plus a hybrid paradigm that trains briefly and then fine-tunes the final spectral layer using a convex negative-Sobolev (H^-1) a posteriori error loss for turbulent Navier-Stokes flows.
Problem
Neural operators for spatiotemporal Navier-Stokes turbulence are expensive to train, lack temporal super-resolution, and reach only low accuracy (about 2-3 digits) relative to the true solution, since end-to-end training minimizes error against numerical solver ground truth.
Approach
Make any FNO variant a trajectory-to-trajectory learner by treating time spectrally and zero-padding the final spectral convolution, then train for ~10 epochs and fine-tune only that final layer with a convex H^-1 (negative Sobolev) residual-based a posteriori error loss computed exactly via Parseval, using an auto-differentiable IMEX solver with a fine time step to get time derivatives.
Key results
- After fine-tuning, the relative L2 error on Taylor-Green vortex drops from 1.94e-01 (after training) to 1.24e-07, with the H^-1 residual dropping to 3.21e-07.
- On the McWilliams turbulence example trained at 64x64 and evaluated at 256x256, L2-trained ST-FNO3d with H^-1 fine-tuning reaches 2.88e-04 L2 error versus 3.87e-02 for FNO3d trained 100 epochs without fine-tuning.
- The authors claim errors up to 10^5 times better than non-fine-tuned FNO variants trained end-to-end.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune ST-FNO3d's spectral layer | ST-FNO3d · Taylor-Green vortex | L2 relative error: 1.94e-01 -> 1.24e-07 | Helped |
| Fine-tune ST-FNO3d's spectral layer | ST-FNO3d · Taylor-Green vortex | H-1 residual norm: 2.18e-01 -> 3.21e-07 | Helped |
| Use H-1 rather than L2 loss for fine-tuning (ablation) | ST-FNO3d trained for 10 ep · Forced turbulence | H-1 residual norm: 2.78e-05 -> 4.59e-07 | Helped |
| Use H-1 rather than L2 loss for fine-tuning (ablation) | ST-FNO3d trained for 10 ep · Forced turbulence | L2 error: 2.82e-04 -> 3.16e-04 | Hurt |
| Fine-tune using H-1 loss instead of L2 loss (ablation) | ST-FNO3d trained with L2 loss · McWilliams isotropic turbulence | L2 error: 1.79e-03 -> 2.88e-04 | Helped |
| Fine-tune using H-1 loss instead of L2 loss (ablation) | ST-FNO3d trained with L2 loss · McWilliams isotropic turbulence | H-1 residual norm: 9.55e-05 -> 4.02e-06 | Helped |
| Train for 100 rather than 10 epochs without fine-tuning | FNO3d · McWilliams isotropic turbulence | L2 error: 4.32e-02 -> 3.87e-02 | Helped |
| Train for 100 rather than 10 epochs without fine-tuning | FNO3d · McWilliams isotropic turbulence | H-1 residual norm: 6.09e-02 -> 2.44e-02 | Helped |
| Fine-tune using H-1 loss rather than no fine-tuning (ablation) | ST-FNO3d trained 10 ep with L2 loss · McWilliams isotropic turbulence | L2 error: 3.69e-02 -> 2.88e-04 | Helped |
| Fine-tune using H-1 loss rather than no fine-tuning (ablation) | ST-FNO3d trained 10 ep with L2 loss · McWilliams isotropic turbulence | H-1 residual norm: 2.35e-02 -> 4.02e-06 | Helped |
Surprising findings
- Fine-tuning only the final spectral layer with an H^-1 loss reaches accuracy that the authors report as better than the ground-truth numerical solver's own error on Taylor-Green vortex (1.24e-07 vs 5.91e-06).
Limitations
- Only FNO-based neural operators benefit from the approach as presented; generalization to other operator learners is left to appendix discussion.
- Fine-tuning requires an auto-differentiable IMEX solver with a fine time step, which is costlier per evaluation.
- Results are shown on 2D periodic turbulence benchmarks only.
Experimental setup
Datasets: 2D isotropic turbulence trajectories generated with pseudo-spectral solver (256^2 downsampled to 64^2) Taylor-Green vortex trajectories
Benchmarks: Taylor-Green vortex 2D isotropic turbulence (forced, original FNO example) McWilliams isotropic turbulence decaying example
Metrics: relative L2 error H^-1 residual norm
Baselines: FNO3d PDE-Refiner PS+RK2 (ground truth solver) end-to-end L2 training H^-1 training
Builds on: Fourier Neural Operator FNO3d PDE-Refiner Physics-informed neural operators
Closest papers by meaning
- Neural Operators with Localized Integral and Differential Kernels (ICML 2024 · similarity 0.89)
- Beyond Regular Grids: Fourier-Based Neural Operators on Arbitrary Domains (ICML 2024 · similarity 0.88)
- Neural Spectral Methods: Self-supervised learning in the spectral domain (ICLR 2024 · similarity 0.88)
- Maximum Update Parametrization and Zero-Shot Hyperparameter Transfer for Fourier Neural Operators (ICML 2025 · similarity 0.87)
- P$^2$C$^2$Net: PDE-Preserved Coarse Correction Network for efficient prediction of spatiotemporal dynamics (NeurIPS 2024 · similarity 0.87)
- Mesh-Free Convolution: Learned Spectral Attenuation and Transport (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.