Research map / Architectures and efficiency / Attention and transformers
Adaptive Physics Transformer with Fused Global-Local Attention for Subsurface Energy Systems
Xin Ju, Hadrian Fung, Yuyan Zhang, Carl Jacquemyn, Matthew D Jackson, Randolph R. Settgast, Sally M Benson, Gege Wen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fuse global perceiver and local GNO branches with gating instead of global only (ablation) | APT · 2D faulted CO2 storage | saturation error dSg (%): 1.50 -> 0.32 | Improved |
| add global branch to local-only GNO encoder via fusion (ablation) | APT · 2D faulted CO2 storage | pressure error dPg (%): 0.18 -> 0.11 | Improved |
| add gated fusion (ablation) | APT · ATES adaptive mesh | relative L2: 0.014 -> 0.011 | Improved |
| add fused attention to GNO-only variant (ablation) | APT · Basin-scale CO2 storage with LGR | saturation error (%): 3.9% -> 2.5% | Improved |
| add gated fusion (ablation) | APT · CarBench super-resolution | score at full resolution: 95.2 -> 96.5 | Improved |
| add gated fusion (ablation) | APT · ATES super-resolution | R2 at full resolution (%): 96.5 -> 97.2 | Improved |
| query at higher resolution than training (ATES) | APT (fused) · ATES super-resolution | R2 (%): 99.0 -> 97.2 | Hurt |
| query at higher resolution than training (CarBench) | APT (fused) · CarBench super-resolution | score: 96.0 -> 96.5 | Improved |
| joint training with Gaussian LGR dataset (ablation) | APT · Channelized CO2 storage | R2 (%): 32.0 -> 86.7 | Improved |
| increase training samples from 40k to 161k (ablation) | APT · ATES data scaling | test loss: 0.1153 -> 0.0812 | Improved |
| increase model size from 0.8M to 69.4M parameters (ablation) | APT · ATES model scaling | test loss: 0.1167 -> 0.0722 | Improved |
| APT versus U-FNO on hydrocarbon pressure | APT · Cartesian hydrocarbon (40x40) | pressure error dP (%): 0.57 -> 0.60 | No clear effect |
Closest papers by meaning
- PDE-Transformer: Efficient and Versatile Transformers for Physics Simulations (ICML 2025 · similarity 0.86)
- Universal Physics Transformers: A Framework For Efficiently Scaling Neural Operators (NeurIPS 2024 · similarity 0.86)
- Geometry Aware Operator Transformer as an efficient and accurate neural surrogate for PDEs on arbitrary domains (NeurIPS 2025 · similarity 0.85)
- SEA: State-Exchange Attention for High-Fidelity Physics Based Transformers (NeurIPS 2024 · similarity 0.84)
- GAIA: Geometry-Adaptive Operator Learning for Forward and Inverse Problems (NeurIPS 2026 · similarity 0.84)
- EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
