Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Learn Singularly Perturbed Solutions via Homotopy Dynamics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Homotopy Dynamics, which trains PINNs on singularly perturbed PDEs by tracing a homotopy path in the PDE parameter epsilon from an easy large-epsilon start to the target small value, with convergence analysis.
Problem
Training neural networks (PINNs and operator learners) for singularly perturbed PDEs is hard because small parameters such as epsilon create near-singular, sharp or high-frequency solutions that make the loss landscape difficult to optimize.
Approach
Treat the PDE parameter epsilon as a homotopy path-tracking parameter: solve the easy large-epsilon problem first, then follow the Davidenko ODE in parameter space (via forward Euler with pseudo-inverse, or a homotopy loss) down to the small target epsilon, with theory linking epsilon to NTK eigenvalues and bounding tracking error.
Key results
- On 2D Allen-Cahn, homotopy dynamics reaches L2RE 8.78e-3 versus 9.56e-1 for original PINN, 8.89e-1 for curriculum, 8.95e-2 for time sequential, and 8.25e-1 for resampling.
- On 1D Allen-Cahn at epsilon=0.01, homotopy dynamics achieves L2RE 8.08e-3 while classical training gives 8.17e-1.
- On high-dimensional Helmholtz (d=20) at epsilon=1/50, homotopy yields L2RE 5.89e-4 versus 9.98e-1 for classical training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use homotopy dynamics instead of classical training | 1D Allen–Cahn · ε=0.03 | L2RE: 1.11 -> 9.83e-3 | Helped |
| Use homotopy dynamics instead of classical training | 1D Allen–Cahn · ε=0.01 | L2RE: 8.17e-1 -> 8.08e-3 | Helped |
| Use homotopy dynamics instead of classical training | 1D Allen–Cahn · ε=0.1 | L2RE: 1.71e-2 -> 1.71e-2 | No clear effect |
| Use homotopy dynamics instead of the strongest listed baseline, Time Seq. | 2D Allen–Cahn · 2D Allen–Cahn | L2RE: 8.95e-2 -> 8.78e-3 | Helped |
| Use homotopy dynamics instead of classical training | Helmholtz, d=20 · ε=1/2 | Relative L2 error: 1.23e-3 -> 5.86e-4 | Helped |
| Use homotopy dynamics instead of classical training | Helmholtz, d=20 · ε=1/20 | Relative L2 error: 7.21e-2 -> 5.00e-4 | Helped |
| Use homotopy dynamics instead of classical training | Helmholtz, d=20 · ε=1/50 | Relative L2 error: 9.98e-1 -> 5.89e-4 | Helped |
| Vary the homotopy initialization along the path, ending at ε=0.01 (ablation) | 1D Allen–Cahn · target ε=0.01 | λ_min(K_ε): 7.38e-8 -> 1.48e-2 | Helped |
Surprising findings
- Table of lambda_min(K) shows a homotopy start at epsilon=0.02 gave the best initialization for epsilon=0.01, suggesting a starting point close to the target is most effective rather than a large epsilon.
Limitations
- Theory uses a two-layer infinite-width NTK-style analysis, while experiments use deeper MLPs.
- Convergence theorem assumes small initial error e_0 and small step sizes, with the Lipschitz constant P_epsilon possibly large as epsilon goes to zero.
- Strategy 1 (pseudo-inverse Euler) is limited to small networks; Strategy 2 requires retraining at each homotopy step.
Experimental setup
Datasets: collocation points sampled from PDE domains (1D/2D Allen-Cahn, high-dimensional Helmholtz, 1D Burgers)
Benchmarks: 1D Allen-Cahn steady state 2D Allen-Cahn steady state high-dimensional Helmholtz (d=20) 1D Burgers steady-state operator learning
Metrics: relative L2 error (L2RE) training loss lambda_min of NTK kernel shock localization MSE
Baselines: Original PINN Curriculum PINN regularization Time sequential training Resampling Xavier initialization Finite difference method
Builds on: PINNs DeepONet Curriculum PINN Regularization neural tangent kernel
Closest papers by meaning
- Number Theoretic Accelerated Learning of Physics-Informed Neural Networks (AAAI 2025 · similarity 0.85)
- Neural Spectral Methods: Self-supervised learning in the spectral domain (ICLR 2024 · similarity 0.85)
- Solving Differential Equations with Constrained Learning (ICLR 2025 · similarity 0.85)
- Breakeven complexity: A new perspective on neural partial differential equation solvers (NeurIPS 2026 · similarity 0.85)
- Physics-Informed Neural Network Policy Iteration: Algorithms, Convergence, and Verification (ICML 2024 · similarity 0.85)
- TENG: Time-Evolving Natural Gradient for Solving PDEs With Deep Neural Nets Toward Machine Precision (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.