Research map / Probabilistic and scientific modeling / Differential equations and PDEs
HEAP: Hyper Extended A-PDHG Operator for Constrained High-dim PDEs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HEAP, a neural operator that unrolls a truncated adaptive PDHG QP solver into a hypernetwork-driven architecture to learn parameter-space evolution for constrained high-dimensional PDEs.
Problem
Neural operators for high-dimensional PDEs struggle to enforce additional equality/inequality constraints beyond the governing equations, causing instability, slow convergence, and constraint violation.
Approach
Formulate the parameter-evolution operator as a QP (via Monte Carlo sampling and Taylor linearization of constraints), then unroll K=3 iterations of adaptive PDHG into a hypernetwork-conditioned architecture with learnable step sizes, momentum, and expansion weights, using differentiable projections.
Key results
- On unconstrained PDEs, HEAP has lower L2 relative PDE residual than CSO at all dimensions tested (e.g., Heat d=5: 0.0041 vs 0.0222).
- On constrained Black-Scholes Type 2 at d=20, HEAP's constraint violation is 0.0165 vs CSO's 0.6936 (reported in the constraint-violation table).
- HEAP costs about 1.6x GPU memory and roughly 2x training/inference time compared with CSO.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HEAP instead of CSO | Neural operator · Unconstrained Heat, Burgers, RD, and Black-Scholes equations, d=5–20 | PDE L2 relative error: Heat: 0.0222 -> 0.0041 (d=5), 0.0218 -> 0.0068 (d=10), 0.0218 -> 0.0106 (d=15), 0.0273 -> 0.0141 (d=20); Burgers: 0.0028 | Helped |
| Use HEAP instead of CSO | Neural operator · Constrained Heat, Burgers, RD, and Black-Scholes equations, d=5–20 | PDE L2 relative error or accumulated residual: Heat: 0.0030 -> 0.0025, 0.0187 -> 0.0044, 0.0136 -> 0.0092, 0.0407 -> 0.0169; Burgers: 0.9798 -> 0.7377, 0.9125 -> 0.628 | Helped |
| Use HEAP instead of CSO | Neural operator · Constrained Heat, Burgers, RD, and Black-Scholes equations, d=5–20 | Physical constraint violation: Heat: 0.2944 -> 0.2540, 0.2843 -> 0.2234, 0.0080 -> 0.0168, 1.58 × 10−4 -> 3.21 × 10−5; Burgers: 0.4544 -> 0.4224, 0.560 | Mixed |
| Use 1 unrolling layer instead of 3 (ablation) | HEAP · Black-Scholes, constraint type 1 | PDE residual: 0.9630 (extended dim 5); 0.8557 (10); 0.9181 (20) | Mixed |
| Use 2 unrolling layers instead of 3 (ablation) | HEAP · Black-Scholes, constraint type 1 | PDE residual: 0.5205 (extended dim 5); 0.5246 (10); 0.4298 (20) | Helped |
| Use 4 unrolling layers instead of 3 (ablation) | HEAP · Black-Scholes, constraint type 1 | PDE residual: 4.5298 (extended dim 5); 1.5917 (10); 1.5857 (20) | Hurt |
| Use extended dimension 5 instead of 20 (ablation) | HEAP · Black-Scholes, constraint type 1 | Constraint residual: 0.5084 (1 layer); 0.1163 (2); 0.4339 (3); 0.3460 (4) | Mixed |
| Use extended dimension 10 instead of 20 (ablation) | HEAP · Black-Scholes, constraint type 1 | Constraint residual: 0.5869 (1 layer); 0.3310 (2); 0.7472 (3); 0.6787 (4) | Mixed |
Surprising findings
- Unrolling more than 3 layers or larger extended dimensions does not consistently improve accuracy in the ablation, with K=3 described as robust.
Limitations
- Runs about 2x slower and uses 1.6x GPU memory compared with the CSO baseline.
- Constraint handling is soft (penalty-based) rather than strictly enforced, and the QP is approximated by Monte Carlo sampling and first-order Taylor expansion.
- Only one baseline (CSO) is compared, and results on several constrained cases remain high in absolute residual.
Experimental setup
Datasets: synthetic PDE initial conditions (Heat, Burgers-type, Reaction-Diffusion, Black-Scholes)
Benchmarks: Heat (unconstrained and constrained) Burgers-type Reaction-Diffusion Black-Scholes
Metrics: L2 relative PDE residual accumulated PDE residual normalized physical constraint violation
Baselines: CSO
Builds on: CSO (Control-based Solution Operator) A-PDHG Neural ODE PDQP
Closest papers by meaning
- Accelerating PDE-Constrained Optimization by the Derivative of Neural Operators (ICML 2025 · similarity 0.88)
- Generalization Error Bounds for Picard-Type Operator Learning in Nonlinear Parabolic PDEs (NeurIPS 2026 · similarity 0.87)
- Boosting Generalization in Parametric PDE Neural Solvers through Adaptive Conditioning (NeurIPS 2024 · similarity 0.87)
- Using Uncertainty Quantification to Characterize and Improve Out-of-Domain Learning for PDEs (ICML 2024 · similarity 0.87)
- Active Learning for Neural PDE Solvers (ICLR 2025 · similarity 0.87)
- Improved Operator Learning by Orthogonal Attention (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.