Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Efficient Generative Transformer Operators for Million-Point PDEs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces ECHO, a hierarchical convolutional encoder-decoder with a DiT flow-matching generative processor that generates full PDE trajectory segments in a compressed spatio-temporal latent space, scaling to million-point grids.
Problem
Neural operators struggle to scale to million-point dense or irregular grids and autoregressive rollouts accumulate error over long horizons for time-dependent PDEs.
Approach
Jointly compress space and time with a hierarchical continuous-convolution encoder-decoder, then generate entire latent trajectories with a flow-matching DiT conditioned on observed latent frames by concatenation, enabling zero-shot forward, inverse, and interpolation tasks.
Key results
- On regular grids, generative ECHO achieves lowest relative MSE on all four systems in Table 1 (e.g., Gray-Scott forward 5.12e-2 vs ENMA 5.44e-2; Rayleigh-Benard inverse 1.16e-1 vs ENMA 1.71e-1).
- On irregular Vorticity forward with 100% points, ECHO reaches 1.71e-1 vs CALM-PDE 4.20e-1 and AROMA 8.42e-1 (forward relative MSE).
- ECHO is the only encoder that runs on 1024x1024 Vorticity super-resolution forecasting (3.88e-1), while all baselines run out of memory.
Surprising findings
- Deterministic FNO and several baselines fail to converge on some datasets, while ECHO remains stable.
- Attention-based AROMA gives the best reconstruction on Cylinder Flow but ECHO surpasses it in forward accuracy.
Limitations
- Encoder-decoder and baselines compared under matched compression, but compression can degrade reconstruction (e.g., Vorticity 20% forward relative MSE is worse than CALM-PDE in some cases)
- Some results use interpolation of sparse observations where ECHO is not uniformly best
Experimental setup
Datasets: Vorticity Shallow-Water Eagle Cylinder Flow Gray-Scott Rayleigh-Benard Acoustic Scattering Maze Active Matter MHD Turbulence Gravity Cooling
Benchmarks: Rayleigh-Benard Gray-Scott Active Matter Acoustic Scattering Maze Vorticity Shallow-Water Eagle Cylinder Flow MHD TGC
Metrics: relative MSE latency compression ratio
Baselines: FNO Transolver++ BCAT AViT ENMA GINO CORAL AROMA CALM-PDE
Builds on: CALM-PDE MM-DiT ENMA AROMA Latent Diffusion Transformers
Closest papers by meaning
- DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training (ICML 2024 · similarity 0.88)
- CALM-PDE: Continuous and Adaptive Convolutions for Latent Space Modeling of Time-dependent PDEs (NeurIPS 2025 · similarity 0.87)
- PDE-Transformer: Efficient and Versatile Transformers for Physics Simulations (ICML 2025 · similarity 0.87)
- AROMA: Preserving Spatial Structure for Latent PDE Modeling with Local Neural Fields (NeurIPS 2024 · similarity 0.87)
- PDE-SSM: A Spectral State Space Approach to Spatial Mixing in Diffusion Transformers (NeurIPS 2026 · similarity 0.86)
- RIGNO: A Graph-based Framework For Robust And Accurate Operator Learning For PDEs On Arbitrary Domains (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.