Research map / Probabilistic and scientific modeling / Differential equations and PDEs

Derivative-Informed Training of Neural Operators On-the-Fly via Sketched Tangent Consistency

NeurIPS 2026 · Probabilistic and scientific modeling › Differential equations and PDEs · approach: Neural operators for general geometries · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes sketched tangent consistency loss (sTCL), an on-the-fly derivative regularizer that enforces the forward sensitivity equation on random input perturbations, with operator-aware conditioning, matching offline derivative-informed FNO training without stored tangent labels.

Problem

Derivative-informed neural-operator training normally needs offline tangent solves and stored derivative labels, which are slow, storage-heavy, and must be regenerated when data, resolution, or sketch basis change.

Approach

Replace precomputed tangent labels with a residual of the forward sensitivity equation evaluated on forward-mode JVPs of the current surrogate along random sketched directions, plus PDE-specific preconditioning (DST inverse-Laplacian residual, short MINRES target matching, Leray-Oseen weighting) chosen by the tangent operator's structure.

Key results

• sTCL/offline-DI function-error ratio below 1.5x in 14/15 cells and below 1.2x in 11/15 cells (e.g., Allen-Cahn N=512 sTCL 0.59% vs offline DI 0.60%)
• On Burgers N=512, Jacobian error 73.53% for sTCL vs 178.08% for offline DI
• Nonlinear diffusion N=1024: DST-preconditioned sTCL lowers function error from 2.98% (pure FNO) to 1.65% and Jacobian error from 10.88% to 6.07%

Surprising findings

• Offline DI has lower function and Jacobian errors than sTCL in most nonlinear diffusion and steady NS cells, so sTCL does not uniformly beat the offline baseline
• Raw forward-sensitivity residual losses square the tangent operator's condition number (kappa(A^T A) = kappa(A)^2), motivating preconditioning

Limitations

• Requires a known, differentiable PDE residual and access to its linearization; black-box simulators are out of scope
• Small hyperparameter grids for lambda; joint tuning of learning rate and lambda not performed
• Steady Navier-Stokes sTCL is substantially slower per job (9990s vs 4810s for offline DI training)

Experimental setup

Datasets: Helmholtz synthetic data nonlinear diffusion-reaction synthetic data Burgers synthetic data Allen-Cahn synthetic data steady Navier-Stokes synthetic data

Benchmarks: Helmholtz Nonlinear diffusion-reaction Burgers Allen-Cahn Steady Navier-Stokes

Metrics: mean per-sample relative function error Jacobian-vector product relative error training wall-clock time

Baselines: Pure FNO Offline DI (DIFNO-style)

Builds on: DIFNO Fourier neural operator Sobolev training physics-informed neural networks Hutchinson trace estimator

Closest papers by meaning

• Accelerating PDE-Constrained Optimization by the Derivative of Neural Operators (ICML 2025 · similarity 0.87)
• Stochastic Taylor Derivative Estimator: Efficient amortization for arbitrary differential operators (NeurIPS 2024 · similarity 0.87)
• Derivative-enhanced Deep Operator Network (NeurIPS 2024 · similarity 0.86)
• MgNO: Efficient Parameterization of Linear Operators via Multigrid (ICLR 2024 · similarity 0.85)
• Guaranteed Approximation Bounds for Mixed-Precision Neural Operators (ICLR 2024 · similarity 0.85)
• Adaptive Coordinate Transforms for Neural Operators (NeurIPS 2026 · similarity 0.85)