AI Paper Atlas

Research map / Architectures and efficiency / Attention and transformers

Scalable Transformer for PDE Surrogate Modeling

Zijie Li, Dule Shu, Amir Barati Farimani

NeurIPS 2023 · Architectures and efficiency › Attention and transformers · approach: Transformer neural operators for PDEs · 20 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace linear attention with factorized attention (ablation)FactFormer · 2D Kolmogorov flow 128x128relative L2 error: 0.5853 -> 0.1529Improved
replace linear attention with factorized attention (ablation)FactFormer · 2D Kolmogorov flow 128x128fwd+bwd time (s): 0.3013 -> 0.0954Improved
replace linear attention with factorized attention (ablation)FactFormer · 2D Kolmogorov flow 128x128peak memory (MB): 12029 -> 5217Improved
use latent marching with pushforward instead of autoregressive training (ablation)FactFormer · 2D Kolmogorov flow 256x256omega avg error: 0.8835 -> 0.1734Improved
use latent marching with pushforward instead of autoregressive training (ablation)FactFormer · 3D smoke buoyancyu final error: 0.7881 -> 0.4482Improved
use latent marching instead of autoregressive training (ablation)FactFormer · 2D Kolmogorov inference timeinference time (s): 3.14 -> 1.38Improved
use latent marching instead of autoregressive training (ablation)F-FNO2D · 2D Kolmogorov flow 256x256omega avg error: 0.1486 -> 0.2453Hurt
use FactFormer instead of Dil-ResNetLM models · 3D isotropic turbulenceu avg error: 0.1250 -> 0.1670Hurt
use FactFormer instead of Dil-ResNetLM models · 3D smoke buoyancyd final error: 0.1062 -> 0.0941Improved
use FactFormer instead of Dil-ResNetLM models · 3D smoke buoyancyinference time (s): 6.94 -> 2.62Improved
use fewer attention heads (ablation)FactFormer · 2D Kolmogorov flow 128x128prediction errorHurt
increase kernel dimension (ablation)FactFormer · 2D Kolmogorov flow 128x128prediction errorImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy