Research map / Time series / Causal and generative time series
Zero-shot Imputation with Foundation Inference Models for Dynamical Systems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains a neural recognition model (Foundation Inference Model) offline on a large synthetic dataset of ODE solutions with noisy sparse observations, then applies it zero-shot to impute missing values in ODE benchmarks and real high-dimensional datasets without fine-tuning.
Problem
Imputing missing values in time series assumed to follow ODE dynamics, where existing methods must be trained from scratch on each target dataset.
Approach
Amortized inference: a DeepONet-style recognition model pretrained on a broad synthetic prior over ODE solutions, observation grids and noise predicts initial conditions and time derivatives, enabling zero-shot imputation; a second model extends context across windows to handle temporal (block) missingness.
Key results
- On ODEBench with 50% point-wise missingness, FIM-ℓ reduces reconstructed-solution MAE to 0.17 vs 1.18 for ODEFormer (no noise).
- FIM-ℓ achieves best MAE on 4 of 8 air-quality/traffic/electricity datasets in zero-shot mode, e.g. Electricity MAE 0.083 vs 0.798 for CSDI.
- On Navier-Stokes with temporal missingness, FIM gets MAE 0.051 (no PCA) vs 0.075 for best spline baseline, outperforming LatentODE's 0.076.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FIM-ℓ instead of ODEFormer for zero-shot inference | ODEBench · ODEBench, γ = 0 | MAE of inferred time derivative: 8.00 ± 0.40 -> 2.44 ± 0.05 | Helped |
| Use FIM-ℓ instead of ODEFormer for zero-shot inference | ODEBench · ODEBench, γ = 0.05 | MAE of inferred time derivative: 7.90 ± 0.60 -> 3.79 ± 0.05 | Helped |
| Use FIM-ℓ instead of ODEFormer for zero-shot inference | ODEBench · ODEBench, γ = 0 | MAE of reconstructed ODE solution: 1.18 ± 0.05 -> 0.17 ± 0.01 | Helped |
| Use FIM-ℓ instead of ODEFormer for zero-shot inference | ODEBench · ODEBench, γ = 0.05 | MAE of reconstructed ODE solution: 1.16 ± 0.05 -> 0.34 ± 0.01 | Helped |
| Use FIM-ℓ instead of the strongest listed baseline | Zero-shot point-wise imputation · Italy | MAE at missing values: 0.285 -> 0.215 | Helped |
| Use FIM-ℓ instead of the strongest listed baseline | Zero-shot point-wise imputation · GuangZhou | MAE at missing values: 2.687 -> 2.427 | Helped |
| Use FIM-ℓ instead of the strongest listed baseline | Zero-shot point-wise imputation · Solar | MAE at missing values: 0.734 -> 0.595 | Helped |
| Use FIM-ℓ instead of the strongest listed baseline | Zero-shot point-wise imputation · Electricity | MAE at missing values: 0.798 -> 0.083 | Helped |
| Use FIM-ℓ instead of the strongest listed baseline | Zero-shot point-wise imputation · Beijing | MAE at missing values: 0.144 -> 0.166 | Hurt |
| Use FIM-ℓ instead of the strongest listed baseline | Zero-shot point-wise imputation · PeMS | MAE at missing values: 0.287 -> 0.365 | Hurt |
| Use FIM-ℓ instead of the strongest listed baseline | Zero-shot point-wise imputation · Pedestrian | MAE at missing values: 0.205 -> 0.273 | Hurt |
| Use FIM-ℓ instead of the strongest listed baseline | Zero-shot point-wise imputation · ETT_h1 | MAE at missing values: 0.223 -> 0.279 | Hurt |
Surprising findings
- FIM-ℓ trained only on additive Gaussian noise handles multiplicative noise well on ODEBench.
- Zero-shot FIM often outperforms state-of-the-art models trained on the target datasets, despite never seeing them.
Limitations
- Performance degrades on empirical data whose dynamics deviate from the synthetic prior (e.g. rapid velocity changes in Van der Pol).
- Instance normalization hides signal-to-noise and scale information; near-constant signals are poorly interpolated.
- Method is limited to imputation; zero-shot forecasting is left to future work.
Experimental setup
Datasets: ODEBench Beijing air quality Italy air quality GuangZhou traffic PeMS Pedestrian Solar ETT_h1 Electricity Motion Capture Navier-Stokes
Benchmarks: ODEBench Beijing Italy GuangZhou PeMS Pedestrian Solar ETT_h1 Electricity Motion Capture Navier-Stokes
Metrics: MAE RMSE R^2 MRE
Baselines: ODEFormer LatentODE BRITS SAITS GP-VAE CSDI BayOTIDE NeuralODEProcesses nPODE
Builds on: DeepONet Neural ODE ODEFormer Neural Processes
Closest papers by meaning
- Zero-shot forecasting of chaotic systems (ICLR 2025 · similarity 0.87)
- A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systems (NeurIPS 2026 · similarity 0.86)
- True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statistics (NeurIPS 2025 · similarity 0.86)
- Out-of-Domain Generalization in Dynamical Systems Reconstruction (ICML 2024 · similarity 0.86)
- LSCD: Lomb--Scargle Conditioned Diffusion for Time series Imputation (ICML 2025 · similarity 0.86)
- Discretizing Continuous Time Series for Imputation with Masked Diffusion Training (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.