Research map / Probabilistic and scientific modeling / Dynamical systems
Foundation Inference Models for Markov Jump Processes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Foundation Inference Models (FIM), a pretrained neural recognition model trained on a synthetic distribution of noisy, sparse Markov jump processes that zero-shot infers rate matrices and initial conditions of unseen MJPs from empirical data.
Problem
Inferring hidden Markov jump processes (rate matrices and initial conditions) from noisy, sparse, coarse-grained observations typically requires training a separate model per dataset, and models overfit to their training set.
Approach
Amortize MJP inference by training one supervised neural recognition model on a broad synthetic prior over MJPs, observation grids and label-noise mechanisms, so that the same pretrained network infers rate matrices and initial distributions for unseen empirical processes with state spaces of different sizes without fine-tuning.
Key results
- On the discrete flashing ratchet, FIM with context c(300,50) performs on par with or better than NeuralMJP, which was trained on the target data, without any fine-tuning.
- On alanine dipeptide, FIM's relaxation time scales (0.611 ns longest) agree in order of magnitude with VampNets (1.920 ns) and NeuralMJP (0.774 ns) baselines.
- Ablation: BiLSTM with learnable query attention reaches RMSE 0.0457 with 300 paths versus 0.193 with a single path.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase the number of paths from 1 to 100 (ablation) | FIM with BiLSTM and learnable query attention · Ablation study | RMSE: 0.193 ± 0.031 -> 0.048 ± 0.011 | Helped |
| Increase the number of paths from 100 to 300 (ablation) | FIM with BiLSTM and learnable query attention · Ablation study | RMSE: 0.048 ± 0.011 -> 0.0457 ± 0.0 | Helped |
| Replace learnable query attention with self-attention (ablation) | FIM with BiLSTM, 1 path · Ablation study | RMSE: 0.193 ± 0.031 -> 0.196 ± 0.031 | Hurt |
| Replace learnable query attention with self-attention (ablation) | FIM with BiLSTM, 100 paths · Ablation study | RMSE: 0.048 ± 0.011 -> 0.049 ± 0.011 | Hurt |
| Replace learnable query attention with self-attention (ablation) | FIM with BiLSTM, 300 paths · Ablation study | RMSE: 0.0457 ± 0.0 -> 0.0458 ± 0.0 | Hurt |
| Replace BiLSTM with Transformer (ablation) | FIM with learnable query attention, 1 path · Ablation study | RMSE: 0.193 ± 0.031 -> 0.197 ± 0.015 | Hurt |
| Replace BiLSTM with Transformer (ablation) | FIM with learnable query attention, 100 paths · Ablation study | RMSE: 0.048 ± 0.011 -> 0.054 ± 0.012 | Hurt |
| Replace BiLSTM with Transformer (ablation) | FIM with learnable query attention, 300 paths · Ablation study | RMSE: 0.0457 ± 0.0 -> 0.0459 ± 0.0 | Hurt |
| Use FIM rather than NeuralMJP for zero-shot simulation | Switching ion channel · Switching ion channel | time-averaged Hellinger distance | Helped |
| Use FIM rather than NeuralMJP for inference | Discrete flashing ratchet · Discrete flashing ratchet | rate-matrix inference | No clear effect |
| Apply FIM to potentials V≥3 | FIM · Discrete flashing ratchet | inference performance | Hurt |
| Use FIM rather than NeuralMJP for zero-shot simulation | Alanine Dipeptide · Alanine Dipeptide | process distance | No clear effect |
Surprising findings
- Averaging predictions over batches of paths yields much lower RMSE (0.0617) than the mean RMSE of individual batches (0.122) on the DFR dataset.
- Performance deteriorates quickly for flashing ratchet potentials V>=3, where rate ratios exceed about three orders of magnitude, outside the synthetic prior.
Limitations
- Performance depends on overlap between the empirical process and the synthetic Beta-based rate prior; large rate ratios (V>=3) degrade accuracy.
- Restricted to bounded state spaces with Beta-distributed rates; power-law or trap-model rate distributions are not covered.
- Ion channel and alanine dipeptide datasets are private and not released, limiting reproducibility on those benchmarks.
Experimental setup
Datasets: synthetic MJP dataset (45K processes) discrete flashing ratchet dataset (5000 paths, from NeuralMJP work) switching ion channel experimental recordings (private communication) all-atom alanine dipeptide simulation (1 microsecond, private communication)
Benchmarks: discrete flashing ratchet switching ion channel alanine dipeptide protein folding model (PFold)
Metrics: RMSE of intensity matrix entries stationary distribution relaxation time scales mean first-passage time entropy production time-averaged Hellinger distance
Baselines: NeuralMJP SDiff (switching diffusion model) VampNets
Builds on: NeuralMJP VampNets Gillespie algorithm
Closest papers by meaning
- Neural Markov Jump Processes (ICML 2023 · similarity 0.91)
- Piecewise deterministic generative models (NeurIPS 2024 · similarity 0.85)
- Mechanistic Interpretability of RNNs emulating Hidden Markov Models (NeurIPS 2025 · similarity 0.85)
- Exact Inference for Continuous-Time Gaussian Process Dynamics (AAAI 2024 · similarity 0.85)
- Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction (ICLR 2025 · similarity 0.85)
- Unlocking Guidance for Discrete State-Space Diffusion and Flow Models (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.