Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometries
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Transolver++ upgrades Transolver's Physics-Attention with local adaptive temperature and Gumbel-softmax slice reparameterization plus a low-communication multi-GPU parallelism scheme, scaling neural PDE solving to million-point meshes.
Problem
Neural PDE solvers are limited to tens of thousands of mesh points, while industrial simulations require million-scale geometries; Transolver suffers from homogeneous physical states and GPU memory bottlenecks at that scale.
Approach
Learn eidetic physical states via a per-point adaptive softmax temperature and Gumbel-softmax slice reparameterization, drop a redundant projection to halve memory, and parallelize Physics-Attention so only M×(C+1) partial sums are all-reduced across GPUs, making communication independent of mesh size.
Key results
- Transolver++ achieves relative L2 of 0.0052 on Elasticity and lowest error on all six standard benchmarks, with an average ~13% relative promotion over the second-best model.
- On DrivAerNet++ Full (2.5M points), Transolver++ reduces volume error to 0.154 and surface error to 0.146 versus Transolver's 0.173 and 0.167 (11.0% and 12.6% relative gains).
- Transolver++ supports 1.2 million mesh points on a single GPU and scales to 2.5M points on 4 A100 GPUs, with communication volume independent of mesh size.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace Transolver with Transolver++ | Transolver · Elasticity | relative L2: 0.0064 -> 0.0052 | Helped |
| replace Transolver with Transolver++ | Transolver · Plasticity | relative L2: 0.0013 -> 0.0011 | Helped |
| replace Transolver with Transolver++ | Transolver · Airfoil | relative L2: 0.0053 -> 0.0048 | Helped |
| replace Transolver with Transolver++ | Transolver · Pipe | relative L2: 0.0033 -> 0.0027 | Helped |
| replace Transolver with Transolver++ | Transolver · NS2d | relative L2: 0.0900 -> 0.0719 | Helped |
| replace Transolver with Transolver++ | Transolver · Darcy | relative L2: 0.0058 -> 0.0049 | Helped |
| add Ada-Temp (ablation) | Transolver · AirCraft | C_L relative L2: 0.037 -> 0.020 | Helped |
| add Speedup (remove f) to Ada-Temp (ablation) | Transolver + Ada-Temp · AirCraft | C_L relative L2: 0.020 -> 0.018 | Helped |
| add Reparameter to Ada-Temp (ablation) | Transolver + Ada-Temp · AirCraft | C_L relative L2: 0.020 -> 0.016 | Helped |
| add Reparameter to Ada-Temp and Speedup (ablation) | Transolver + Ada-Temp + Speedup · AirCraft | C_L relative L2: 0.018 -> 0.014 | Helped |
Surprising findings
- Transolver's slice weights become increasingly homogeneous in deeper layers, which motivated the eidetic-state design.
- Baseline GNNs and several neural operators were reported to degenerate or be unable to handle million-scale meshes, so they were evaluated on subsampled and reconstructed meshes.
Limitations
- Baselines that cannot handle million-scale meshes were evaluated on subsampled and KNN-reconstructed meshes, which may disadvantage them.
- The AirCraft dataset is newly introduced but its release is not stated; industrial evaluation uses very few test cases (10).
Experimental setup
Datasets: Elasticity Plasticity Airfoil Pipe NS2d Darcy DrivAerNet++ AirCraft
Benchmarks: Elasticity Plasticity Airfoil Pipe NS2d Darcy DrivAerNet++ AirCraft
Metrics: relative L2 drag coefficient error lift coefficient error R-squared GPU memory running time
Baselines: Transolver GNOT LNO 3D-GeoCA GINO Galerkin Geo-FNO GraphSAGE PointNet Graph U-Net MeshGraphNet OFormer
Builds on: Transolver Gumbel-softmax FNO Ring Attention DeepSpeed-Ulysses
Closest papers by meaning
- Transolver: A Fast Transformer Solver for PDEs on General Geometries (ICML 2024 · similarity 0.93)
- Text2PDE: Latent Diffusion Models for Accessible Physics Simulation (ICLR 2025 · similarity 0.88)
- Breakeven complexity: A new perspective on neural partial differential equation solvers (NeurIPS 2026 · similarity 0.88)
- PhyMPGN: Physics-encoded Message Passing Graph Network for spatiotemporal PDE systems (ICLR 2025 · similarity 0.87)
- INC: An Indirect Neural Corrector for Auto-Regressive Hybrid PDE Solvers (NeurIPS 2025 · similarity 0.87)
- Unisolver: PDE-Conditional Transformers Towards Universal Neural PDE Solvers (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.