Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Sub-Sequential Physics-Informed Learning with State Space Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PINNMamba, a data-free physics-informed learner that uses a time-varying selective SSM (Mamba) with overlapping sub-sequence contrastive alignment to propagate initial-condition information and counter simplicity bias in PINNs.
Problem
PINNs fail to propagate initial-condition patterns over time (gradual distortion, over-smoothed solutions) on hard PDEs such as convection, reaction, and wave equations, due to continuous-discrete mismatch and simplicity bias.
Approach
Model each collocation point's temporal sub-sequence with a time-varying Selective SSM (Mamba-style) so the discrete recurrence approximates continuous dynamics, and align overlapping sub-sequence predictions with a contrastive/consistency loss so diverse predictions agree, removing simplicity bias.
Key results
- On convection, PINNMamba reaches rMAE 0.0188 versus 0.4527 for PINNsFormer and 0.8514 for PINN; on wave it reaches 0.0197 vs 0.1433 for KAN.
- Abstract claims error reductions of up to 86.3% relative to the state-of-the-art architecture.
- Removing sub-sequence alignment raises convection rMAE from 0.0188 to 0.1436, and removing SSM entirely raises it to 0.7785.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PINNMamba instead of the strongest listed baseline | Convection · Convection | rMAE: 0.4527 -> 0.0188 | Helped |
| use PINNMamba instead of the strongest listed baseline | Reaction · Reaction | rMAE: 0.0146 -> 0.0094 | Helped |
| use PINNMamba instead of the strongest listed baseline | Wave · Wave | rMAE: 0.1433 -> 0.0197 | Helped |
| add sub-sequence alignment (ablation) | PINNMamba · Convection, Reaction, and Wave | rMAE: 0.1436 -> 0.0188; 0.0291 -> 0.0094; 0.0481 -> 0.0197 | Helped |
| use sub-sequences instead of a long sequence of the entire domain (ablation) | PINNMamba · Convection, Reaction, and Wave | rMAE: 0.6492 -> 0.0188; 0.6731 -> 0.0094; 0.3391 -> 0.0197 | Helped |
| use a time-varying SSM instead of a linear time-invariant SSM (ablation) | PINNMamba · Convection, Reaction, and Wave | rMAE: 0.0241 -> 0.0188; 0.0179 -> 0.0094; 0.0664 -> 0.0197 | Helped |
| use SSM instead of MLP (ablation) | PINNMamba · Convection, Reaction, and Wave | rMAE: 0.7785 -> 0.0188; 0.9836 -> 0.0094; 0.3341 -> 0.0197 | Helped |
| use Wavelet activation instead of Tanh (ablation) | PINNMamba · Convection, Reaction, and Wave | rMAE: 0.4531 -> 0.0188; 0.0299 -> 0.0094; 0.3151 -> 0.0197 | Helped |
| set sub-sequence length to 3 instead of 7 (ablation) | PINNMamba · Convection, Reaction, and Wave | rMAE: 0.6698; 0.0150; 0.5288 | Mixed |
| set sub-sequence length to 5 instead of 7 (ablation) | PINNMamba · Convection, Reaction, and Wave | rMAE: 0.0092; 0.0131; 0.0278 | Mixed |
| set sub-sequence length to 9 instead of 7 (ablation) | PINNMamba · Convection, Reaction, and Wave | rMAE: 0.0410; 0.0105; 0.0343 | Hurt |
| add RoPINN | PINNMamba · Convection and Wave | rMAE: 0.0188 -> 0.0102; 0.0197 -> 0.0121 | Helped |
Surprising findings
- Replacing the time-varying selective SSM with a linear time-invariant SSM gives a smaller degradation on convection than removing alignment, while removing SSM entirely causes severe failure.
- PINNMamba and PINNsFormer hit out-of-memory on many PINNacle problems even on an H100 GPU.
Limitations
- Sequence-based design runs out of memory on many PINNacle problems, so only 6 of 16 were tested.
- Sub-sequence length and interval are sensitive hyperparameters; performance varies strongly (e.g., length 3 or 21 fails on convection).
Experimental setup
Datasets: 1-D convection PDE collocation points 1-D reaction PDE collocation points 1-D wave PDE collocation points 2-D Navier-Stokes collocation points PINNacle benchmark problems
Benchmarks: 1-D convection 1-D reaction 1-D wave 2-D Navier-Stokes PINNacle
Metrics: rMAE rRMSE training loss memory overhead training time
Baselines: PINN QRes PINNsFormer KAN RoPINN gPINN vPINN
Builds on: Physics-Informed Neural Networks Mamba (selective SSM) PINNsFormer S4 / HiPPO Wavelet activation
Closest papers by meaning
- Efficient Error Certification for Physics-Informed Neural Networks (ICML 2024 · similarity 0.91)
- PINNsFormer: A Transformer-Based Framework For Physics-Informed Neural Networks (ICLR 2024 · similarity 0.91)
- RoPINN: Region Optimized Physics-Informed Neural Networks (NeurIPS 2024 · similarity 0.90)
- Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspective (NeurIPS 2025 · similarity 0.89)
- Physics-Informed Neural Network Policy Iteration: Algorithms, Convergence, and Verification (ICML 2024 · similarity 0.89)
- Per-Loss Adapters for Gradient Conflict in Physics-Informed Neural Networks (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.