Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Physics-informed Temporal Alignment for Auto-regressive PDE Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PITA, a self-supervised physics-informed temporal alignment framework that uses sparse-regression PDE discovery on auto-regressive predictions to counter shortcut-driven error accumulation in PDE foundation models.
Problem
Auto-regressive PDE foundation models suffer from a shortcut problem (copying previous frames) that causes error accumulation on long trajectories and out-of-distribution downstream tasks.
Approach
Discovers governing PDE coefficients via sparse regression from both ground-truth and predicted windows, then uses the discrepancy between discovered coefficients as a consistency loss plus a physics residual loss, combined with data loss via uncertainty-based weighting and alternating direction optimization; requires no known physics priors.
Key results
- On long-trajectory tasks PITA reduces total accumulated error by 30.22% on average over auto-regressive finetuning.
- For small models (Ti, S) PITA achieves best results on 75% of short-trajectory tasks with an average 11.21% improvement.
- On the out-of-distribution Burgers downstream task, PITA gives an average 37.8% error reduction over auto-regressive finetuning.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add physics loss to data-loss training (ablation) | PITA · FNO-NS-1e-3, Task 1 | nRMSE: 0.00301 -> 0.00166 | Helped |
| Add consistency loss to data-loss training (ablation) | PITA · FNO-NS-1e-3, Task 1 | nRMSE: 0.00301 -> 0.00165 | Helped |
| Add consistency loss alongside physics loss (ablation) | PITA · FNO-NS-1e-3, Task 1 | nRMSE: 0.00166 -> 0.00157 | Helped |
| Use spatial downsampling factor l=4 instead of l=1 or l=2 (ablation) | PITA · FNO-NS-1e-3, Task 2 | nRMSE: 0.00169 -> 0.00157; 0.00168 -> 0.00157 | Helped |
| Set retained temporal frames TC=7 instead of TC=10, TC=5, or TC=3 (ablation) | PITA · FNO-NS-1e-3, Task 3 | nRMSE: 0.00161 -> 0.00155; 0.00170 -> 0.00155; 0.00157 -> 0.00155 | Mixed |
| Replace adaptive weighting with manually chosen loss weights (ablation) | PITA · FNO-NS-1e-3, Task 4 | nRMSE: 0.00157 -> 0.00172; 0.00157 -> 0.00175; 0.00157 -> 0.00171 | Hurt |
| Add second-order diffusion terms to the one-order library (ablation) | PITA · Burgers, Task 5 | nRMSE: 0.00287 -> 0.00152 | Helped |
| Add first- and second-order derivatives to the library (ablation) | PITA · Burgers, Task 5 | nRMSE: 0.00325 -> 0.00152 | Helped |
| Add PITA temporal alignment | DPOT-S · FNO-NS-1e-4, short trajectory | nRMSE: 0.00641 -> 0.00232 | Helped |
| Add PITA temporal alignment | DPOT-S · PDEBench-DR, long trajectory | nRMSE: 0.01350 -> 0.00994 | Helped |
| Add PITA temporal alignment | DPOT-S · PDEBench-SWE, long trajectory | nRMSE: 0.00235 -> 0.00137 | Helped |
| Add PITA temporal alignment | DPOT-S · Burgers downstream | nRMSE: 0.00533 -> 0.00458 | Helped |
Surprising findings
- Finetuning pretrained foundation models can be worse than training from scratch on out-of-distribution tasks such as PDEArena-SWE and Burgers.
- Larger downsampling factors (fewer spatial grid points) often gave better performance, attributed to reduced overfitting.
Limitations
- Extra computational cost from gradient propagation through the two additional losses.
- Performance depends on the completeness of the candidate function library and the noise level of observations.
- Some large-model settings (FNO-NS-1e-3/1e-4 with DPOT-L) were less stable under the baseline training settings.
Experimental setup
Datasets: FNO-NS-1e-5 FNO-NS-1e-4 FNO-NS-1e-3 PDEBench-CNS PDEBench-SWE PDEBench-DR PDEArena-NS PDEArena-NS-Force CFDBench-NS Burgers (viscous)
Benchmarks: FNO-NS-1e-5 FNO-NS-1e-4 FNO-NS-1e-3 PDEBench-CNS PDEBench-SWE PDEBench-DR PDEArena-NS PDEArena-NS-Force CFDBench-NS Burgers
Metrics: nRMSE
Baselines: DPOT MPP FNO auto-regressive finetuning
Builds on: DPOT PINN DeepMoD PINN-SR
Closest papers by meaning
- Poseidon: Efficient Foundation Models for PDEs (NeurIPS 2024 · similarity 0.86)
- Learning a Neural Solver for Parametric PDEs to Enhance Physics-Informed Methods (ICLR 2025 · similarity 0.86)
- INC: An Indirect Neural Corrector for Auto-Regressive Hybrid PDE Solvers (NeurIPS 2025 · similarity 0.86)
- Stable Long-Horizon PDE Forecasting via Latent Structured Spectral Propagators (NeurIPS 2026 · similarity 0.86)
- Physics-informed Reduced Order Modeling of Time-dependent PDEs via Differentiable Solvers (NeurIPS 2025 · similarity 0.85)
- Text2PDE: Latent Diffusion Models for Accessible Physics Simulation (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.