Research map / Probabilistic and scientific modeling / Dynamical systems

OrthoReg: Orthogonal Regularization for Hybrid Symbolic-Neural Dynamical Systems

NeurIPS 2026 · Probabilistic and scientific modeling › Dynamical systems · approach: Switching and law-discovery dynamics · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes OrthoReg, which penalizes empirical inner products between a neural augmentation and a sparse symbolic library so the residual cannot absorb symbolic structure in hybrid ODE models.

Problem

Hybrid symbolic-neural dynamical models: when the symbolic library is learned via sparse (L1) discovery, standard L2 regularization of the neural augmentation fails to prevent the neural part from relearning symbolic structure, yielding redundant, uninterpretable models.

Approach

Replace the APHYNITY-style L2 penalty on the neural augmentation with a penalty on its squared empirical inner products with each library function, directly enforcing orthogonality to the symbolic span so the decomposition is complementary (in-library vs. orthogonal complement).

Key results

• On the damped pendulum, OrthoReg reaches F1 0.93 vs 0.61 for L2 hybrid and 0.43 for pure symbolic, with 3.6 terms vs 6.8 and 11.4.
• OrthoReg reduces OOD-T3 derivative MSE on the pendulum to 0.02 (x1e2 scale) vs 7.90 for L2 hybrid and 34.00 for pure symbolic, at the cost of higher in-distribution state MSE (3.65 vs 0.31 for L2).
• On the Duffing oscillator, OrthoReg achieves the lowest cross-basin OOD-T2 state MSE (4.93 vs 7.46 for L2 and 11.03 for pure) and F1 0.63 vs 0.59.

Surprising findings

• OrthoReg sacrifices in-distribution state MSE (3.65e-2 vs 0.31e-2 for L2 hybrid on the pendulum) in exchange for symbolic recovery and OOD gains.
• Pure symbolic regression achieves the lowest in-distribution derivative MSE on the pendulum and the Duffing ID MSE, so the neural residual does not uniformly help accuracy.

Limitations

• Guarantees are stated under the empirical inner product and fixed differentiable libraries; population-level guarantees for the learned augmentation are left open.
• Gains diminish under heavy observation noise and when missing dynamics are strongly entangled with library features.
• Mini-batch estimation introduces O(1/B) bias, so experiments use full- or large-batch estimates.

Experimental setup

Datasets: simulated damped pendulum trajectories Duffing oscillator trajectories Lotka-Volterra trajectories time-modulated SIR trajectories

Benchmarks: modified damped pendulum Duffing oscillator Lotka-Volterra time-modulated SIR

Metrics: normalized MSE (derivative and state) F1 for symbolic support recovery number of symbolic terms

Baselines: Pure symbolic (SINDy-style L1) Hybrid L2 (APHYNITY-style) PINN Universal ODE PySINDy STLSQ

Builds on: APHYNITY SINDy Universal ODEs

Closest papers by meaning

• Interpretable Relational Inference with LLM-Guided Symbolic Dynamics Modeling (NeurIPS 2026 · similarity 0.86)
• Out-of-Domain Generalization in Dynamical Systems Reconstruction (ICML 2024 · similarity 0.86)
• Identifying latent state transitions in non-linear dynamical systems (ICLR 2025 · similarity 0.85)
• eXponential FAmily Dynamical Systems (XFADS): Large-scale nonlinear Gaussian state-space modeling (NeurIPS 2024 · similarity 0.85)
• Almost-Linear RNNs Yield Highly Interpretable Symbolic Codes in Dynamical Systems Reconstruction (NeurIPS 2024 · similarity 0.85)
• Deep-Koopman-KANDy: Dictionary Discovery for Deep-Koopman Operators with Kolmogorov-Arnold Networks for Dynamics (NeurIPS 2026 · similarity 0.85)