Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Diffusion Tempering Improves Parameter Estimation with Probabilistic Integrators for Ordinary Differential Equations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes diffusion tempering, which progressively lowers the diffusion (kappa) hyperparameter of the Fenrir probabilistic ODE-integrator likelihood during gradient-based parameter estimation to avoid local minima.
Problem
Gradient-based parameter estimation in ODEs is hampered by many local minima and sensitivity to initial conditions, so optimizers often fail to recover true parameters, especially for complex models like Hodgkin-Huxley.
Approach
Treat the diffusion hyperparameter of a probabilistic integrator (Fenrir) as a regularization/tempering schedule rather than learning it jointly: start with a high diffusion (smooth loss surface) and successively lower it, warm-starting each optimization from the previous estimate.
Key results
- For the pendulum, diffusion tempering converged in 100% of 100 initializations versus 99% for Fenrir and 32% for RK least-squares.
- On the six-parameter HH1 model, tempered Fenrir had far more reliable recovery than RK; the abstract states it obtains reliable estimates with a practically relevant number of parameters.
- For the two-compartment six-parameter HH model (HH2 with 6 parameters), diffusion tempering recovered parameters correctly in 88% of runs while RK converged in 0%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace jointly learned diffusion with a decreasing diffusion schedule | Fenrir parameter optimization · Pendulum (PD) | CONV: 0.99 -> 1.00 | Helped |
| Replace jointly learned diffusion with a decreasing diffusion schedule | Fenrir parameter optimization · Lotka–Volterra, 2 parameters | CONV: 0.20 -> 0.77 | Helped |
| Replace jointly learned diffusion with a decreasing diffusion schedule | Fenrir parameter optimization · HH₁, 3 parameters | CONV: 0.51 -> 0.99 | Helped |
| Replace jointly learned diffusion with a decreasing diffusion schedule | Fenrir parameter optimization · HH₂, 6 parameters | CONV: 0.50 -> 0.88 | Helped |
| Replace RK least-squares optimization with diffusion tempering | ODE parameter estimation · HH₂, 6 parameters | CONV: 0.00 -> 0.88 | Helped |
| Use early stopping during tempering except at the final diffusion value (ablation) | Diffusion tempering (ours+) · HH₁, 2 parameters | ITER: 696.22 ± 78.10 -> 197.81 ± 40.38 | Helped |
| Use early stopping during tempering except at the final diffusion value (ablation) | Diffusion tempering (ours+) · HH₁, 2 parameters | CONV: 1.00 -> 0.96 | Hurt |
| Use an exponential rather than linear log-diffusion schedule (ablation) | Diffusion tempering · HH₁, 2 parameters | computation cost: around 30% | Helped |
| Use a steeper, shorter schedule between log κ=20 and log κ=0 (ablation) | Diffusion tempering · HH₁, 2 parameters | iterations: about 20% | Helped |
| Use more than 16 tempering steps (ablation) | Diffusion tempering · HH₁, 2 parameters | correctly inferred parameters | No clear effect |
| Fix diffusion at a low value instead of the likelihood-optimal diffusion at true parameters (ablation) | Fenrir with fixed diffusion · HH, 2 parameters | CONV | Helped |
| Fix diffusion at one value rather than adaptively tempering it (ablation) | Fenrir parameter optimization · HH, 2 parameters | CONV | Hurt |
Surprising findings
- Learning the diffusion jointly (default Fenrir) performed on par with RK least-squares for the two-parameter HH model, offering no benefit.
- The kappa maximizing likelihood at the true parameters performed worse than fixing kappa at a low value.
Limitations
- Diffusion tempering is the most expensive method in runtime due to sequential optimizations and cubic scaling in ODE dimension.
- Assumes observation noise variance is known.
- Not applicable to neural ODEs yet since adjoints are not derived for probabilistic solvers.
Experimental setup
Datasets: synthetic ODE trajectories (pendulum, Lotka-Volterra, Hodgkin-Huxley)
Benchmarks: pendulum Lotka-Volterra Hodgkin-Huxley single- and multi-compartment models
Metrics: trajectory RMSE (tRMSE) relative parameter RMSE (pRMSE) convergence fraction (CONV) number of iterations (ITER) number of correctly identified parameters
Baselines: Runge-Kutta least-squares (RK) Fenrir Fenrir with fixed kappa
Builds on: Fenrir Runge-Kutta least-squares regression probabilistic numerical ODE solvers
Closest papers by meaning
- Tortoise and Hare Guidance: Accelerating Diffusion Model Inference with Multirate Integration (NeurIPS 2025 · similarity 0.86)
- Enhanced Diffusion Sampling via Extrapolation with Multiple ODE Solutions (ICLR 2025 · similarity 0.86)
- Closing the Approximation Gap in Simulation-free Latent SDEs (NeurIPS 2026 · similarity 0.85)
- Learning non-equilibrium diffusions with Schrödinger bridges: from exactly solvable to simulation-free (NeurIPS 2025 · similarity 0.85)
- Predicting partially observable dynamical systems via diffusion models with a multiscale inference scheme (NeurIPS 2025 · similarity 0.85)
- Arbitrarily-Conditioned Multi-Functional Diffusion for Multi-Physics Emulation (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.