Research map / Time series / Causal and generative time series
Log Neural Controlled Differential Equations: The Lie Brackets Make A Difference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Log-NCDEs, which apply the Log-ODE method with Lie brackets of a neural CDE vector field (depth-2, Lip(gamma) regularised FCNN) to train NCDEs, outperforming NCDEs, NRDEs, LRU, S5, S6 and MAMBA on UEA and PPG-DaLiA.
Problem
Neural CDEs lag state-of-the-art sequence models (S5, LRU, MAMBA) on long multivariate time series classification and regression, and NRDEs approximate the Lie bracket structure only via a costly wide neural network output.
Approach
Construct the Log-ODE vector field from iterated Lie brackets of a NCDE's own neural vector field f_theta (rather than a wide network output as in NRDEs), enforce Lip(gamma) regularity via spectral-norm-style weight penalty, and compute brackets efficiently with batched Jacobian-vector products.
Key results
- Log-NCDE achieves the best average test accuracy on the six UEA-MTSCA datasets (64.3 vs 62.0 for the next best, S6) and best average rank (2.1).
- On PPG-DaLiA, Log-NCDE attains the lowest test MSE (9.56 x10^-2 ± 0.59), ahead of MAMBA (10.65) and NRDE (9.90).
- Log-NCDE has lower average run time (2128 s) than NCDE (6650 s) and NRDE (7284 s) on UEA-MTSCA, though all four stacked recurrent models are faster.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Construct the Log-ODE vector field using iterated Lie brackets of the NCDE vector field instead of parameterizing it as a neural network | NRDE · UEA-MTSCA average | accuracy: 60.6 -> 64.3 | Helped |
| Construct the Log-ODE vector field using iterated Lie brackets of the NCDE vector field instead of parameterizing it as a neural network | NRDE · PPG-DaLiA test | MSE (× 10⁻²): 9.90\pm 0.97 -> 9.56 \pm 0.59 | Helped |
| Use Log-NCDE rather than NCDE | NCDE · UEA-MTSCA average | run time (s): 6649.91 -> 2128.32 | Helped |
| Use Log-NCDE rather than NRDE | NRDE · UEA-MTSCA average | run time (s): 7284.20 -> 2128.32 | Helped |
| Use Log-NCDE rather than NRDE | NRDE · UEA-MTSCA datasets | accuracy | Helped |
| Use Log-NCDE rather than NCDE | NCDE · EigenWorms test | accuracy: 75.0 \pm 3.9 -> 85.6 \pm 5.1 | Helped |
| Use Log-NCDE rather than NRDE | NRDE · EthanolConcentration test | accuracy: 25.3 \pm 1.8 -> 34.4 \pm 6.4 | Helped |
| Use Log-NCDE rather than LRU | LRU · Heartbeat test | accuracy: 78.4 \pm 6.7 -> 75.2 \pm 4.6 | Hurt |
| Use Log-NCDE rather than S5 | S5 · SelfRegulationSCP1 test | accuracy: 89.9 \pm 4.6 -> 83.1 \pm 2.8 | Hurt |
| Use Log-NCDE rather than LRU | LRU · EigenWorms test | accuracy: 87.8 \pm 2.8 -> 85.6 \pm 5.1 | Hurt |
Surprising findings
- Stacked recurrent models (LRU, S5, S6) struggle on the toy signature-classification tasks as the number of channels in the label term grows, while MAMBA is the only one that performs well on the four-channel task.
- MAMBA has the lowest average accuracy on UEA-MTSCA despite being a strong state-of-the-art model.
Limitations
- Only depth-1 and depth-2 Log-ODE approximations are considered; theory for Lip(gamma) bounds exists only up to gamma<=2.
- Computing iterated Lie brackets via JVPs grows rapidly with depth N and dimension v.
- NCDEs require sequential ODE solving, preventing parallelisation over time, and all stacked recurrent models remain faster in run time.
Experimental setup
Datasets: Toy dataset (synthetic, 100k series) UEA-MTSCA subset: EigenWorms, EthanolConcentration, Heartbeat, MotorImagery, SelfRegulationSCP1, SelfRegulationSCP2 PPG-DaLiA
Benchmarks: UEA-MTSCA (six datasets) PPG-DaLiA Toy signature-classification dataset
Metrics: test accuracy mean squared error average rank GPU memory run time
Baselines: NCDE NRDE LRU S5 S6 MAMBA
Builds on: neural rough differential equations neural controlled differential equations S5 linear recurrent unit MAMBA
Closest papers by meaning
- Neural Rough Differential Equations for Long Time Series (ICML 2021 · similarity 0.91)
- Controlled Differential Equations on Long Sequences via Non-standard Wavelets (ICML 2023 · similarity 0.91)
- On the Generalization and Approximation Capacities of Neural Controlled Differential Equations (ICLR 2024 · similarity 0.89)
- Faithful Embeddings of Irregular and Asynchronous Data for Online Log-NCDEs (NeurIPS 2026 · similarity 0.85)
- Symbolic Neural Ordinary Differential Equations (AAAI 2025 · similarity 0.84)
- Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Data (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.