Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Poseidon: Efficient Foundation Models for PDEs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Poseidon pretrains a multiscale SwinV2-based operator transformer (scOT) with time-conditioned layer norms and an all2all semigroup-based training loss on fluid-dynamics trajectories, then finetunes it on 15 out-of-distribution PDE tasks.
Problem
Operator learning for PDEs is sample-inefficient; the paper asks whether pretraining on a small set of PDEs can yield a foundation model that generalizes to unseen PDEs with few task-specific samples.
Approach
A hierarchical multiscale vision transformer (scOT) with lead-time-conditioned layer norms predicts full trajectories continuously in time, and an all2all loss uses semigroup property to pair all time-point couples within a trajectory, giving O(K^2) training samples per trajectory.
Key results
- Poseidon-L needs a median of about 20 task-specific samples to match FNO trained on 1024 samples, with an order-of-magnitude fewer samples on 13 of 15 tasks.
- At 128 samples (512 for time-independent), Poseidon-L has a mean accuracy gain of about an order of magnitude over FNO.
- Poseidon outperforms CNO-FM on 14 of 15 tasks and is best on 14 of 15 tasks overall, including 8 of 9 unseen-PDE tasks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Pretrain Poseidon rather than train a model from scratch | PDE downstream tasks · 15 downstream tasks | accuracy gain (AG): Poseidon-L is the best performing model on 14 of the 15 tasks | Helped |
| Use the Poseidon transformer foundation model instead of CNO-FM | PDE downstream tasks · 15 downstream tasks | efficiency gain (EG): approximately 100 task-specific examples -> approximately 20 | Helped |
| Increase Poseidon model size | Poseidon · downstream tasks | test error | Helped |
| Increase pretraining dataset size | Poseidon-B · 9 of 15 downstream tasks | accuracy | Helped |
| Reduce diversity by dropping trajectories for CE-CRP, CE-Gauss, and NS-Sines (ablation) | Poseidon-B · downstream tasks | test error | Hurt |
| Freeze latent-space parameters during finetuning (ablation) | Poseidon · Poisson-Gauss | solution-operator accuracy | No clear effect |
Surprising findings
- Poseidon generalizes to Poisson-Gauss, an elliptic Dirichlet problem unrelated to the pretraining fluid dynamics, and performs well with few samples.
- Freezing the latent space and finetuning only embeddings still performs very effectively on downstream tasks.
Limitations
- Only a few PDEs and data distributions were used in pretraining, with no elliptic PDEs or non-Cartesian geometries in pretraining.
- Evaluated on a limited set of 15 tasks; generality as a PDE foundation model remains to be shown broadly.
Experimental setup
Datasets: NS-Sines NS-Gauss CE-RP CE-CRP CE-KH CE-Gauss PDEBench
Benchmarks: 15 downstream PDE tasks (PDEgym) NS-PwC NS-SVS NS-BB NS-SL CE-RPUI Poisson-Gauss Helmholtz ACE Wave-Gauss Wave-Layer
Metrics: relative L1 error at final time accuracy gain (AG) efficiency gain (EG)
Baselines: FNO CNO scOT CNO-FM MPP-B DPOT
Builds on: SwinV2 Fourier Neural Operator Convolutional Neural Operator ConvNeXt
Closest papers by meaning
- CALM-PDE: Continuous and Adaptive Convolutions for Latent Space Modeling of Time-dependent PDEs (NeurIPS 2025 · similarity 0.88)
- RIGNO: A Graph-based Framework For Robust And Accurate Operator Learning For PDEs On Arbitrary Domains (NeurIPS 2025 · similarity 0.87)
- On the Benefits of Memory for Modeling Time-Dependent PDEs (ICLR 2025 · similarity 0.87)
- Data-Efficient Operator Learning via Unsupervised Pretraining and In-Context Learning (NeurIPS 2024 · similarity 0.87)
- Physics-informed Reduced Order Modeling of Time-dependent PDEs via Differentiable Solvers (NeurIPS 2025 · similarity 0.87)
- MultiPDENet: PDE-embedded Learning with Multi-time-stepping for Accelerated Flow Simulation (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.