Research map / Probabilistic and scientific modeling / Differential equations and PDEs

Per-Loss Adapters for Gradient Conflict in Physics-Informed Neural Networks

NeurIPS 2026 · Probabilistic and scientific modeling › Differential equations and PDEs · approach: Physics-informed neural network training · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a 1000-step gradient-conflict profiling workflow for PINNs that chooses between scalar loss reweighting and per-loss low-rank adapters on a shared trunk, validated over 60+ PDE configurations.

Problem

PINN training with multiple physics/data losses suffers from gradient conflict, but existing fixes (scalar loss balancing vs. full-parameter gradient surgery) lack a rule for when each is appropriate.

Approach

Treat PINN gradient conflict as regime-dependent: profile a short unmodified run to measure directional conflict vs. magnitude imbalance, and when persistent directional conflict is found, attach one low-rank adapter per loss to a shared trunk to create loss-indexed parameter subspaces, combined with reweighting (FAMO/GradNorm).

Key results

• On 10K-epoch Burgers, FAMO+UAM and GN+UAM reach about 4.6e-3 relative L2 error, a 1.8x improvement over Vanilla.
• On Helmholtz, GN+UAM reaches 1.0e-3 relative L2 error, a 28x improvement over Vanilla (Vanilla 2.8e-2).
• Adapters add 15% to 41% parameter overhead; on inverse problems, full-parameter gradient surgery methods (PCGrad, CAGrad, NashMTL, ConFIG) diverged due to ~10^4x gradient norm mismatch.

Surprising findings

• Full-parameter-space gradient surgery methods produce invalid values within the first 50 to 200 epochs on inverse problems.
• For K=3 inverse problems, FAMO and GradNorm without adapters outperform adapter combinations by 10% to 25%.

Limitations

• Gains are regime-dependent; adapters can hurt on magnitude-dominated problems such as K=3 inverse problems and K=5/K=6 multi-physics systems.
• Profiled ratio R is only an empirical surrogate for the theoretical quantity U_K, and the theorem analyzes a fixed spherical model.
• Blockwise NTK analysis uses an idealized exact-separation surrogate; the implemented architecture retains off-diagonal adapter couplings.

Experimental setup

Datasets: Burgers Helmholtz Allen-Cahn Convection-diffusion Klein-Gordon Darcy flow inverse Burgers inverse heat inverse Poisson thermoelastic reactive transport magnetohydrodynamics

Benchmarks: Burgers Helmholtz Allen-Cahn Convection-diffusion Klein-Gordon Darcy flow inverse Burgers inverse heat inverse Poisson thermoelastic reactive transport magnetohydrodynamics

Metrics: relative L2 error average rank 95% confidence interval Cohen's d

Baselines: Vanilla PINN FAMO GradNorm Causal training Self-adaptive weights ConFIG PCGrad CAGrad NashMTL AlignedMTL

Builds on: PINNs FAMO GradNorm PCGrad LoRA

Closest papers by meaning

• ConFIG: Towards Conflict-free Training of Physics Informed Neural Networks (ICLR 2025 · similarity 0.92)
• Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspective (NeurIPS 2025 · similarity 0.92)
• Challenges in Training PINNs: A Loss Landscape Perspective (ICML 2024 · similarity 0.90)
• PINNsFormer: A Transformer-Based Framework For Physics-Informed Neural Networks (ICLR 2024 · similarity 0.89)
• Sub-Sequential Physics-Informed Learning with State Space Model (ICML 2025 · similarity 0.89)
• Dual Cone Gradient Descent for Training Physics-Informed Neural Networks (NeurIPS 2024 · similarity 0.89)