Research map / Architectures and efficiency / Attention and transformers
Multiscale Attention Wavelet Neural Operator for Capturing Steep Trajectories in Biochemical Systems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MAWNO, a physics-informed wavelet neural operator with multiscale self-attention on wavelet detail coefficients, to solve biochemical ODEs with steep trajectories without labeled data.
Problem
Neural operators such as FNO and WNO struggle to approximate biochemical ODE systems with abrupt, steep trajectory switches (e.g., MPF, Brusselator, Goodwin models).
Approach
Apply self-attention to the high-frequency wavelet detail coefficients at multiple decomposition scales inside a wavelet neural operator block, and train with an ODE residual plus initial-condition loss without ground-truth labels.
Key results
- On the MPF model, MAWNO achieves NMSE of about 2e-7 to 1e-6 versus 2e-5 to 5e-5 for WNO, and FNO and PINN fail.
- On the Brusselator, MAWNO achieves NMSE of 2e-6 to 4e-6, against about 2e-5 for FNO and 1e-5 to 1e-4 for WNO.
- On Goodwin, FNO, WNO and MAWNO all reach NMSE around 1.3e-6, while PINN ranges 1e-5 to 5e-5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MAWNO instead of PINN | Biochemical trajectory prediction · Goodwin | NMSE: 1 × 10−5 ∼5 × 10−5 vs approximately 1.3×10−6 | Helped |
| Use MAWNO instead of FNO or WNO | Biochemical trajectory prediction · Goodwin | NMSE: approximately 1.3×10−6 | No clear effect |
| Use MAWNO instead of PINN | Biochemical trajectory prediction · Brusselator | NMSE: 2×10−6 ∼4×10−6 | Helped |
| Use MAWNO instead of FNO | Biochemical trajectory prediction · Brusselator | NMSE: approximately 2 × 10−5 vs 2×10−6 ∼4×10−6 | Helped |
| Use MAWNO instead of WNO | Biochemical trajectory prediction · Brusselator | NMSE: 1 × 10−5 ∼ 1 × 10−4 vs 2×10−6 ∼4×10−6 | Helped |
| Use MAWNO instead of PINN or FNO | Biochemical trajectory prediction · MPF | NMSE: 2 × 10−7 ∼ 1 × 10−6 | Helped |
| Use MAWNO instead of WNO | Biochemical trajectory prediction · MPF | NMSE: 2 × 10−5 ∼ 5 × 10−5 vs 2 × 10−7 ∼ 1 × 10−6 | Helped |
| Increase Brusselator trajectory steepness | MAWNO · Brusselator, varying steepness | NMSE | Helped |
| Increase wavelet decomposition layers (ablation) | MAWNO on Brusselator · Brusselator | training time | Hurt |
| Increase wavelet basis support moments (ablation) | MAWNO on Brusselator · Brusselator | training time | Hurt |
| Increase training dataset size (ablation) | MAWNO on Brusselator · Brusselator | training time | Hurt |
| Vary the number of MAWNO blocks (ablation) | MAWNO on MPF · MPF | NMSE | Mixed |
Surprising findings
- PINN fails completely on the Brusselator model, and FNO and PINN fail on MPF, while WNO remains workable there.
Limitations
- Evaluated only on three low-dimensional ODE biochemical systems with a single-run-style NMSE comparison.
- Training requires very many epochs (50k to 500k), and larger wavelet levels or depths increase training time and overfitting risk.
- Performance degrades beyond a certain MAWNO block depth (degradation after four layers on MPF).
Experimental setup
Datasets: Goodwin ODE system Brusselator ODE system MPF ODE system
Benchmarks: Goodwin Brusselator MPF
Metrics: NMSE
Baselines: PINN FNO WNO
Builds on: Wavelet Neural Operator Fourier Neural Operator Physics-Informed Neural Networks Transformer self-attention
Closest papers by meaning
- Hierarchical Gradient-Based Genetic Sampling for Accurate Prediction of Biological Oscillations (AAAI 2025 · similarity 0.81)
- Operator-Learning-Inspired Modeling of Neural Ordinary Differential Equations (AAAI 2024 · similarity 0.80)
- In-Context Multi-Operator Learning with DeepOSets (NeurIPS 2026 · similarity 0.80)
- Wavelet Diffusion Neural Operator (ICLR 2025 · similarity 0.80)
- Disentangling the Roles of Distinct Cell Classes with Cell-Type Dynamical Systems (NeurIPS 2024 · similarity 0.80)
- BioOSS: A Bio-Inspired Oscillatory State System with Spatio-Temporal Dynamics (NeurIPS 2025 · similarity 0.79)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.