Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Transolver: A Fast Transformer Solver for PDEs on General Geometries
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Transolver replaces standard attention with Physics-Attention, which softly assigns mesh points to learnable physics slices, encodes slice tokens, and applies attention among tokens, achieving linear complexity and state-of-the-art PDE solving on six benchmarks and two industrial car/airfoil tasks.
Problem
Transformers applied directly to large unstructured PDE meshes are computationally expensive (quadratic attention) and struggle to capture intricate physical correlations among massive individual mesh points.
Approach
Learn soft slice assignments of mesh points from their features so that points with similar physical states share a slice; aggregate each slice into a physics-aware token, run attention among the M tokens (linear in mesh size), and deslice back to mesh points.
Key results
- Transolver achieves state-of-the-art on all six standard benchmarks, with relative error reductions of 10.2% to 29.7% over the second-best model (e.g., 0.0064 vs 0.0086 on Elasticity).
- On Shape-Net Car, Transolver reaches drag coefficient relative L2 of 0.0103 and Spearman 0.9935, versus 0.0159 and 0.9842 for the second-best 3D-GeoCA.
- On AirfRANS out-of-distribution Reynolds and angle splits, Transolver achieves lift coefficient error 0.2996 and 0.1500, the best among compared models.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Physics-Attention instead of attention over mesh points | PDE benchmarks · Elasticity | Relative L2: 0.0086 -> 0.0064 | Helped |
| use Physics-Attention instead of attention over mesh points | PDE benchmarks · Plasticity | Relative L2: 0.0017 -> 0.0012 | Helped |
| use Physics-Attention instead of attention over mesh points | PDE benchmarks · Airfoil | Relative L2: 0.0059 -> 0.0053 | Helped |
| use Physics-Attention instead of attention over mesh points | PDE benchmarks · Pipe | Relative L2: 0.0047 -> 0.0033 | Helped |
| use Physics-Attention instead of attention over mesh points | PDE benchmarks · Navier–Stokes | Relative L2: 0.1195 -> 0.0900 | Helped |
| use Physics-Attention instead of attention over mesh points | PDE benchmarks · Darcy | Relative L2: 0.0065 -> 0.0057 | Helped |
| use Physics-Attention rather than the strongest reported baseline | Shape-Net Car · Shape-Net Car | C_D Relative L2: 0.0159 -> 0.0103 | Helped |
| use Physics-Attention rather than the strongest reported baseline | AirfRANS · AirfRANS | C_L Relative L2: 0.1476 -> 0.1030 | Helped |
| set number of slices M to 1 instead of 64 (ablation) | Transolver · Elasticity | Relative L2: 0.0064 -> 0.0148 | Hurt |
| increase number of slices from M=8 to M=64 (ablation) | Transolver · Elasticity | Relative L2: 0.0071 -> 0.0064 | Helped |
| increase number of slices from M=16 to M=64 (ablation) | Transolver · Darcy | Relative L2: 0.0067 -> 0.0059 | Helped |
| increase number of slices from M=64 to M=1024 (ablation) | Transolver · Elasticity | Relative L2: 0.0064 -> 0.0068 | Hurt |
Surprising findings
- Learned attention among physics-aware tokens is much sharper than linear attention among mesh points (KL divergence from uniform 1.78 vs 0.38 on Elasticity).
- Replacing learnable slices with fixed 4x4 regular squares hurts performance even on Darcy, which is originally a regular grid.
Limitations
- Performance depends on the number of slices M, which must be tuned per PDE (too large M fragments the physics domain).
- Baselines like geo-FNO degrade on complex unstructured meshes, and some Transformer baselines (ONO, OFormer) are unstable on large meshes, so comparisons on industrial tasks are limited.
- Authors note that large-scale pre-training of Transolver as a foundation PDE solver is not yet explored.
Experimental setup
Datasets: Elasticity Plasticity Airfoil Pipe Navier-Stokes Darcy Shape-Net Car AirfRANS
Benchmarks: Elasticity Plasticity Airfoil Pipe Navier-Stokes Darcy Shape-Net Car AirfRANS
Metrics: relative L2 drag coefficient relative L2 lift coefficient relative L2 Spearman rank correlation GPU memory running time per epoch
Baselines: FNO geo-FNO U-NO LSM WMT GNOT FactFormer ONO OFormer Galerkin Transformer HT-Net GINO
Builds on: Transformer Fourier neural operator Galerkin Transformer geo-FNO
Closest papers by meaning
- Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometries (ICML 2025 · similarity 0.93)
- PDE-Transformer: Efficient and Versatile Transformers for Physics Simulations (ICML 2025 · similarity 0.89)
- Universal Physics Transformers: A Framework For Efficiently Scaling Neural Operators (NeurIPS 2024 · similarity 0.86)
- UGM2N: An Unsupervised and Generalizable Mesh Movement Network via M-Uniform Loss (NeurIPS 2025 · similarity 0.86)
- Geometry Aware Operator Transformer as an efficient and accurate neural surrogate for PDEs on arbitrary domains (NeurIPS 2025 · similarity 0.85)
- GeoTransolver: Learning Physics on Irregular Domains using Multi-scale Geometry Aware Physics Attention Transformer (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.