Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Understanding Generalization in Physics Informed Models through Affine Variety Dimensions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proves that generalization of physics-informed linear regression in hybrid data+physics settings is governed by the dimension of the affine variety defined by the physical constraints (not parameter count), with an approximation method and experiments on ODEs/PDEs.
Problem
Existing theory for physics-informed models assumes complete physical knowledge and is mostly limited to linear equations, leaving the effect of incomplete physics combined with observational data on generalization for nonlinear systems unclear.
Approach
A unified residual form (trial functions plus measures) unifies collocation and variational constraints; the feasible weight set for a linear-in-basis model is an affine variety, and the minimax risk bound depends on its dimension d_V instead of the number of basis functions d. The dimension is estimated by sampling feasible weights and computing the rank of the Jacobian of constraint polynomials.
Key results
- On the discrete linear Bernoulli equation (d=100 and 200, d_V=1), PILR achieves test MSE about 0.012 versus about 0.48 to 0.63 for ridge regression.
- On the discrete nonlinear diffusion equation (d=2010 and 4020, d_V=10 and 20), PILR reaches test MSE 0.26 and 0.22 versus about 1.1 for ridge regression.
- With basis misspecification (omitted analytical frequency), test MSE stays around 1.43 regardless of basis size, showing approximation error dominates.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PILR instead of RR | Discrete linear Bernoulli, h=1/100 · Discrete linear Bernoulli (h=1/100) | Test MSE: 0.48 \pm 0.32 -> 0.012 \pm 0.0025 | Helped |
| Use PILR instead of RR | Discrete linear Bernoulli, h=1/200 · Discrete linear Bernoulli (h=1/200) | Test MSE: 0.63 \pm 0.43 -> 0.011 \pm 0.0013 | Helped |
| Use PILR instead of RR | Discrete nonlinear Bernoulli, h=1/100 · Discrete nonlinear Bernoulli (h=1/100) | Test MSE: 0.60 \pm 0.41 -> 0.013 \pm 0.0024 | Helped |
| Use PILR instead of RR | Discrete nonlinear Bernoulli, h=1/200 · Discrete nonlinear Bernoulli (h=1/200) | Test MSE: 0.72 \pm 0.49 -> 0.013 \pm 0.0018 | Helped |
| Use PILR instead of RR | Discrete linear diffusion, (h_t, h_x)=(1/400, 2/10) · Discrete linear diffusion (h_t=1/400, h_x=2/10) | Test MSE: 2.21 \pm 0.56 -> 1.13 \pm 0.30 | Helped |
| Use PILR instead of RR | Discrete linear diffusion, (h_t, h_x)=(1/400, 2/20) · Discrete linear diffusion (h_t=1/400, h_x=2/20) | Test MSE: 2.14 \pm 0.57 -> 0.79 \pm 0.16 | Helped |
| Use PILR instead of RR | Discrete nonlinear diffusion, (h_t, h_x)=(1/200, 2/10) · Discrete nonlinear diffusion (h_t=1/200, h_x=2/10) | Test MSE: 1.12 \pm 0.40 -> 0.26 \pm 0.11 | Helped |
| Use PILR instead of RR | Discrete nonlinear diffusion, (h_t, h_x)=(1/200, 2/20) · Discrete nonlinear diffusion (h_t=1/200, h_x=2/20) | Test MSE: 1.11 \pm 0.40 -> 0.22 \pm 0.10 | Helped |
| Increase the basis size from d=17 to d=33 while omitting the known analytical frequency (ablation) | PILR on the harmonic oscillator · Harmonic oscillator (10 data points; misspecified basis) | Test MSE: 1.435 \pm 0.646 -> 1.434 | No clear effect |
Surprising findings
- When the basis omits the true frequency, increasing the basis size from 17 to 33 leaves test MSE essentially unchanged, so physics constraints cannot compensate for approximation error.
Limitations
- Analysis centers on linear regression over fixed bases; neural networks and ELMs are not analyzed.
- Bound constants can be loose relative to minimax-optimal rates, and beta is only bounded via Petrovskii-Oleinik-Milnor.
- Requires sufficient expressive capacity of the basis; misspecified bases dominate error.
Experimental setup
Datasets: simulated harmonic oscillator data simulated diffusion equation data finite-difference solutions of Bernoulli and diffusion equations
Benchmarks: harmonic oscillator diffusion equation discrete Bernoulli equation discrete diffusion equation
Metrics: test MSE
Baselines: ridge regression
Builds on: PINN physics-informed kernel effective dimension Dudley entropy integral
Closest papers by meaning
- ConFIG: Towards Conflict-free Training of Physics Informed Neural Networks (ICLR 2025 · similarity 0.85)
- The Challenges of the Nonlinear Regime for Physics-Informed Neural Networks (NeurIPS 2024 · similarity 0.85)
- Learning a Neural Solver for Parametric PDEs to Enhance Physics-Informed Methods (ICLR 2025 · similarity 0.85)
- Improving Energy Natural Gradient Descent through Woodbury, Momentum, and Randomization (NeurIPS 2025 · similarity 0.85)
- Neural Oscillators for Generalization of Physics-Informed Machine Learning (AAAI 2024 · similarity 0.85)
- An operator preconditioning perspective on training in physics-informed machine learning (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.