Research map / Architectures and efficiency / Attention and transformers
GNOT: A General Neural Operator Transformer for Operator Learning
Zhongkai Hao, Zhengyi Wang, Hang Su, Chengyang Ying, Yinpeng Dong, Songming Liu, Ze Cheng, Jian Song, Jun Zhu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add self-attention after cross-attention (cross+self block) instead of cross+cross (ablation) | GNOT attention block · NACA, Elasticity, NS2d-c (p) | relative L2 error: NACA 3.52e-2 -> 7.57e-3; Elasticity 3.31e-2 -> 8.65e-3; NS2d-c p 1.50e-2 -> 7.41e-3 | Improved |
| Put self-attention before cross-attention (self+cross) instead of cross+self (ablation) | GNOT attention block · NS2d-c (p) | relative L2 error: 7.41e-3 -> 9.89e-2 | Hurt |
| Use 3 experts in geometric gating instead of 1 expert (ablation) | GNOT · Heat | relative L2 error: 0.04212 -> 0.03695 | Improved |
| Use 8 or 16 experts instead of 3 (ablation) | GNOT · Heat | relative L2 error: 0.03695 -> 0.04732 (8 experts) | Hurt |
| Increase number of attention heads from 1 to 16 (with 1 expert) (ablation) | GNOT · Heat | relative L2 error: 0.04131 (1 head) vs 0.04180 (4 heads) vs 0.03952 (16 heads) | Mixed |
| Heterogeneous normalized linear attention with geometric gating (GNOT) vs baselines on NS2d-c, NACA, Inductor2d | GNOT · NS2d-c, NACA, Inductor2d | relative L2 error: e.g., NACA 1.38e-2 (Geo-FNO) -> 7.57e-3 | Improved |
| GNOT vs GK-Transformer on Darcy2d (uniform grid) | GNOT · Darcy2d | relative L2 error: 1.05e-2 vs 8.40e-3 (GK-Transformer) | Hurt |
Later papers that compared with this method
3 later accepted papers report GNOT as a baseline in their results tables; 2 of them report a better number on most of the results they share with it.
- SAOT: An Enhanced Locality-Aware Spectral Transformer for Solving PDEs (AAAI 2026 · better on 2 of 2 shared results)
- Improved Operator Learning by Orthogonal Attention (ICML 2024 · better on 0 of 1 shared result)
- Positional Knowledge is All You Need: Position-induced Transformer (PiT) for Operator Learning (ICML 2024 · better on 1 of 1 shared result)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Geometry Aware Operator Transformer as an efficient and accurate neural surrogate for PDEs on arbitrary domains (NeurIPS 2025 · similarity 0.88)
- Data-Efficient Operator Learning via Unsupervised Pretraining and In-Context Learning (NeurIPS 2024 · similarity 0.88)
- MgNO: Efficient Parameterization of Linear Operators via Multigrid (ICLR 2024 · similarity 0.87)
- Improved Operator Learning by Orthogonal Attention (ICML 2024 · similarity 0.87)
- In-Context Multi-Operator Learning with DeepOSets (NeurIPS 2026 · similarity 0.87)
- Optimal deep learning of holomorphic operators between Banach spaces (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
