Research map / Probabilistic and scientific modeling / Differential equations and PDEs
ConFIG: Towards Conflict-free Training of Physics Informed Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ConFIG, a conflict-free gradient combination for multi-loss training (with an alternating-momentum variant M-ConFIG), evaluated on PINNs and CelebA multi-task learning.
Problem
Gradient conflicts between multiple additive loss terms (PDE residuals vs. initial/boundary conditions in PINNs, and many tasks in multi-task learning) cause slow or stuck training.
Approach
Compute an update direction whose dot product with every loss-specific gradient is positive, via the pseudoinverse of the normalized gradient matrix with uniform direction weights and an adaptive magnitude equal to the sum of projections; M-ConFIG reuses alternating momentum buffers so only one loss gradient is backpropagated per iteration.
Key results
- On PINN two-loss and three-loss scenarios (Burgers, Schrödinger, Kovasznay, Beltrami), ConFIG consistently outperforms the Adam baseline and PCGrad in most cases, with PCGrad/IMTL-G differing in some cases.
- On the 40-task CelebA multi-task benchmark, ConFIG and M-ConFIG 30 yield the best mean rank and average F1 score among compared methods.
- M-ConFIG reduces per-iteration cost to about 0.56 of the three-loss ConFIG cost (r=1.67, speed-up 1.67/3) and achieves lower test MSE at fixed wall time.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ConFIG instead of Adam | PINNs with two loss terms · Burgers, Schrödinger, Kovasznay, and Beltrami flow | test MSE relative improvement over Adam | Helped |
| use ConFIG instead of PCGrad | PINNs with two loss terms · Burgers, Schrödinger, Kovasznay, and Beltrami flow | test MSE relative improvement over Adam | Helped |
| use IMTL-G instead of Adam | PINNs with two loss terms · Schrödinger | test MSE relative improvement over Adam | Hurt |
| use ConFIG instead of other methods | PINNs with three loss terms · Beltrami flow | test MSE | Helped |
| use PCGrad instead of other methods | PINNs with three loss terms · Burgers and Schrödinger | test MSE | Helped |
| use M-ConFIG instead of other methods at equal wall-clock time | PINNs · two- and three-loss scenarios | test MSE | Helped |
| set equal direction weights rather than alternative weighting strategies (ablation) | ConFIG · PINNs with two or three loss terms | test MSE relative improvement over Adam | Helped |
| set equal direction weights rather than ReLoBRaLo weights (ablation) | ConFIG · Kovasznay flow with two losses | test MSE relative improvement over Adam | Hurt |
| increase the number of tasks (ablation) | M-ConFIG · CelebA MTL | average F1 | Hurt |
| use ConFIG instead of M-ConFIG (ablation) | CelebA MTL with 10 tasks and equal training time · CelebA | average F1: 0.536 -> 0.635 | Helped |
| increase M-ConFIG gradient updates per iteration to 20 (ablation) | M-ConFIG · CelebA MTL with 40 tasks | training time relative to ConFIG: 56% | Helped |
Surprising findings
- PCGrad outperforms ConFIG on the Burgers and Schrödinger three-loss cases, and IMTL-G beats ConFIG on Kovasznay and Beltrami two-loss cases while falling below Adam on Schrödinger.
- Earlier momentum-per-loss variants (MA-ConFIG / MultiAdam-style second momentum) were found inaccurate and numerically unstable compared with the proposed alternating scheme.
Limitations
- M-ConFIG performance degrades as the number of loss terms/tasks grows, requiring more momentum updates per iteration.
- Memory cost of storing per-loss momentum grows with parameter count and number of losses.
- Does not resolve other PINN difficulties such as chaotic problems or causality issues.
Experimental setup
Datasets: CelebA
Benchmarks: CelebA
Metrics: MSE relative improvement over Adam mean rank average F1 score wall-clock time
Baselines: Adam PCGrad IMTL-G LRA MinMax ReLoBRaLo LS UW DWA GradDrop CAGrad RLW
Builds on: PCGrad IMTL-G Adam ReLoBRaLo
Closest papers by meaning
- Per-Loss Adapters for Gradient Conflict in Physics-Informed Neural Networks (NeurIPS 2026 · similarity 0.92)
- Challenges in Training PINNs: A Loss Landscape Perspective (ICML 2024 · similarity 0.91)
- Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspective (NeurIPS 2025 · similarity 0.90)
- Dual Cone Gradient Descent for Training Physics-Informed Neural Networks (NeurIPS 2024 · similarity 0.89)
- Sub-Sequential Physics-Informed Learning with State Space Model (ICML 2025 · similarity 0.89)
- RoPINN: Region Optimized Physics-Informed Neural Networks (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.