Research map / Architectures and efficiency / Attention and transformers
Simple yet Effective: Low-Rank Spatial Attention for Neural Operators
Zherui Yang, Haiyang Xin, Tao Du, Ligang Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add latent self-attention instead of MLP in latent processing (ablation) | LRSA · Navier-Stokes | relative L2 error (x10^-2): 5.44 -> 4.84 | Improved |
| decouple compression and reconstruction projections (vs symmetric tying) (ablation) | LRSA · Navier-Stokes | relative L2 error (x10^-2): 5.93 -> 4.84 | Improved |
| decouple compression and reconstruction projections (vs symmetric tying) (ablation) | LRSA · Elasticity | relative L2 error (x10^-2): 0.47 -> 0.33 | Improved |
| increase latent rank M (ablation) | LRSA · simple static fields | relative L2 error | No clear effect |
| increase latent rank M (ablation) | LRSA · Navier-Stokes | relative L2 error | Improved |
| use LRSA with standard attention primitives instead of Transolver slicing in FP16 | Transolver baseline · mixed precision (Airfoil, NS, ShapeNet Car) | training stability | Improved |
| LRSA-FP16 vs Transolver-FP32 | ShapeNet Car training · ShapeNet Car (N ~ 32k) | training latency / memory: 3.2x lower latency, 2.1x lower peak memory | Improved |
| LRSA vs HPM with very little training data | LRSA · Navier-Stokes, 200 training samples | relative L2 error (x10^-2): HPM 18.1 vs LRSA 26.2 | Hurt |
| learnable latent bottleneck instead of Laplace-eigenfunction geometric priors | LRSA vs HPM · Irregular Darcy | relative L2 error: 7.36e-3 -> 7.56e-3 | No clear effect |
| LRSA instead of Transolver under resolution shift | LRSA · Airfoil zero-shot resolution (111x51) | relative L2 error (x10^-2): 7.850 -> 0.468 | Improved |
Closest papers by meaning
- Neural Interpretable PDEs: Harmonizing Fourier Insights with Attention for Scalable and Interpretable Physics Discovery (ICML 2025 · similarity 0.88)
- Improved Operator Learning by Orthogonal Attention (ICML 2024 · similarity 0.87)
- Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEs (NeurIPS 2024 · similarity 0.87)
- AROMA: Preserving Spatial Structure for Latent PDE Modeling with Local Neural Fields (NeurIPS 2024 · similarity 0.87)
- Latent Neural Operator for Solving Forward and Inverse PDE Problems (NeurIPS 2024 · similarity 0.87)
- CALM-PDE: Continuous and Adaptive Convolutions for Latent Space Modeling of Time-dependent PDEs (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
