Research map / Probabilistic and scientific modeling / Differential equations and PDEs
CoLoRA: Continuous low-rank adaptation for reduced implicit neural modeling of parameterized partial differential equations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CoLoRA, a continuous low-rank adaptation neural reduced model for parameterized PDEs whose time-dependent latent scalings adapt pre-trained low-rank weights, offering a purely data-driven (CoLoRA-D) and a Galerkin-based equation-driven (CoLoRA-EQ) online phase.
Problem
Rapidly predicting PDE solution fields over time for new physics parameters and initial conditions is expensive with classical solvers, and linear model reduction suffers from the Kolmogorov barrier for transport-dominated problems; operator learning needs many training trajectories.
Approach
Pre-train an MLP whose hidden layers are LoRA-style low-rank updates scaled by a time- and parameter-dependent scalar alpha(t, mu) produced by a hyper-network; only the low-dimensional latent state phi(t, mu) is adapted online, either by evaluating the hyper-network (data-driven) or by solving a Neural Galerkin / Dirac-Frenkel least-squares ODE (equation-driven).
Key results
- CoLoRA-D reaches relative errors around 1e-3 with about 10 training trajectories on Vlasov and Burgers', while F-FNO is roughly one order of magnitude worse (e.g., Vlasov 7.48e-2 vs 2.37e-3 at 10 trajectories).
- On the 2D wave benchmark CoLoRA-D achieves MSE 1.891e-7 with 7,505 parameters, versus DINo at 9.495e-6 with 579,776 parameters.
- CoLoRA-D is more than four orders of magnitude faster than the full-order models, and CoLoRA-EQ still gives about two orders of magnitude speedup.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CoLoRA-D instead of CORAL | 3D spherical shallow water forecasting · 3D spherical shallow water | MSE: 3.44e-6 vs 3.19e-06 | Helped |
| Use CoLoRA-D instead of DINo | 2D wave forecasting · 2D wave | MSE: 9.495e-6 vs 1.891e-07 | Helped |
| Use CoLoRA-D instead of F-FNO | Vlasov forecasting with 10 trajectories · Vlasov, 10 trajectories | relative error: 7.48e-2 vs 2.37e-3 | Helped |
| Use CoLoRA-D instead of F-FNO | 2D Burgers forecasting with 10 trajectories · 2D Burgers, 10 trajectories | relative error: 2.40e-2 vs 1.76e-3 | Helped |
| Use CoLoRA-D instead of F-FNO | RDE forecasting with 10 trajectories · RDE, 10 trajectories | relative error: 5.69e-3 vs 4.47e-4 | Helped |
| Use CoLoRA-D instead of CoLoRA-EQ | Vlasov forecasting · Vlasov, 100 trajectories | relative error: 1.58e-3 vs 9.87e-4 | Helped |
| Use CoLoRA-D instead of CoLoRA-EQ | 2D Burgers forecasting · 2D Burgers, 100 trajectories | relative error: 2.27e-3 vs 4.96e-4 | Helped |
| Use CoLoRA-D instead of CoLoRA-EQ | RDE forecasting · RDE, 100 trajectories | relative error: 1.49e-3 vs 2.05e-4 | Helped |
| Increase the number of latent parameters beyond about 2 (ablation) | CoLoRA · Vlasov, Burgers, and RDE | relative error | No clear effect |
| Use 2–3 latent parameters rather than a larger latent state (ablation) | CoLoRA · Vlasov, Burgers, and RDE | relative error | Helped |
Surprising findings
- The purely data-driven CoLoRA-D achieves slightly lower error than the equation-driven CoLoRA-EQ in the reported experiments.
- Linear interpolation over space, time, and parameter becomes accurate in high-data regimes for sufficiently smooth problems.
Limitations
- Theoretical analysis of approximation properties is limited; the Kolmogorov-barrier result is shown only for linear advection.
- Pre-training once is insufficient for problems with bifurcations not represented in training data; online adaptation of offline parameters is needed.
- Extrapolation beyond the in-distribution range of parameters and time is not well handled by the hyper-network.
Experimental setup
Datasets: Vlasov simulations (own FOM) 2D Burgers simulations (own FOM) Rotating detonation wave simulations (own FOM) DINo 2D wave benchmark CORAL 3D spherical shallow water benchmark
Benchmarks: Vlasov 2D Burgers RDE 2D wave (DINo benchmark) 3D spherical shallow water (CORAL benchmark)
Metrics: mean relative error MSE runtime speedup parameter count mass conservation
Baselines: F-FNO DINo CORAL MP-PDE DeepONet linear projection (POD-type best approximation) linear interpolation
Builds on: LoRA Neural Galerkin schemes Dirac-Frenkel variational principle F-FNO DINo
Closest papers by meaning
- Physics-informed Reduced Order Modeling of Time-dependent PDEs via Differentiable Solvers (NeurIPS 2025 · similarity 0.85)
- Boosting Generalization in Parametric PDE Neural Solvers through Adaptive Conditioning (NeurIPS 2024 · similarity 0.85)
- Learning a Neural Solver for Parametric PDEs to Enhance Physics-Informed Methods (ICLR 2025 · similarity 0.84)
- Solving Differential Equations with Constrained Learning (ICLR 2025 · similarity 0.84)
- CALM-PDE: Continuous and Adaptive Convolutions for Latent Space Modeling of Time-dependent PDEs (NeurIPS 2025 · similarity 0.84)
- DISCO: learning to DISCover an evolution Operator for multi-physics-agnostic prediction (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.