Research map / Time series / Foundation models for time series
MIRA: Medical Time Series Foundation Model for Real-World Health Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MIRA, a decoder-only medical time series foundation model with continuous-time RoPE, frequency-specific MoE, and a Neural ODE extrapolation block, pretrained on 454B medical time points and continued from Time-MoE.
Problem
Generalist time series foundation models assume regular sampling and struggle with the irregular intervals, heterogeneous sampling rates, and missing values of medical time series, and existing medical TSFMs focus on classification rather than continuous-time forecasting.
Approach
Extends RoPE to continuous timestamps (CT-RoPE), adds a sparse frequency-specific mixture-of-experts with a shared expert, and uses a Neural ODE block to evolve latent states to arbitrary target timestamps, pretrained on a large curated medical corpus continued from Time-MoE.
Key results
- MIRA_large achieves the best RMSE on several out-of-distribution datasets, with the paper claiming over 8% average RMSE reduction versus the strongest baselines.
- In-distribution zero-shot, MIRA_large gets the best RMSE on 4 of 5 pretraining datasets (SleepEDF, PTB-XL, MIMIC-III, MIMIC-IV, WAVES) per the table counts.
- Ablation: removing the CT-Extrapolation Block increases RMSE and MAE by about 5% and 8%, the largest drop among the components.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add CT-RoPE (ablation) | MIRA_base · MIRA ablation evaluation | RMSE: 0.158 -> 0.154 | Helped |
| add CT-RoPE (ablation) | MIRA_base · MIRA ablation evaluation | MAE: 0.125 -> 0.118 | Helped |
| add MoE Block (ablation) | MIRA_base · MIRA ablation evaluation | RMSE: 0.157 -> 0.154 | Helped |
| add MoE Block (ablation) | MIRA_base · MIRA ablation evaluation | MAE: 0.122 -> 0.118 | Helped |
| add CT-Extrapolation Block (ablation) | MIRA_base · MIRA ablation evaluation | RMSE: 0.162 -> 0.154 | Helped |
| add CT-Extrapolation Block (ablation) | MIRA_base · MIRA ablation evaluation | MAE: 0.128 -> 0.118 | Helped |
| use Top-1 instead of Top-2 expert routing (ablation) | MIRA_base · MIRA ablation evaluation | RMSE: 0.154 -> 0.160 | Hurt |
| use Top-4 instead of Top-2 expert routing (ablation) | MIRA_base · MIRA ablation evaluation | RMSE: 0.154 -> 0.154 | No clear effect |
| use Top-6 instead of Top-2 expert routing (ablation) | MIRA_base · MIRA ablation evaluation | RMSE: 0.154 -> 0.156 | Hurt |
| use Top-8 instead of Top-2 expert routing (ablation) | MIRA_base · MIRA ablation evaluation | RMSE: 0.154 -> 0.159 | Hurt |
| increase missing rate from 10%+ to 90%+ | MIRA_large · WHO FluNet | RMSE: 2.85 -> 3.97 | Hurt |
Surprising findings
- Smaller MIRA variants surpass larger general-domain foundation models, suggesting data relevance matters more than model size.
- MIRA_large outperforms the full-shot fine-tuned MIT-BIH baseline while still trailing full-shot models on CDC-IHA and several other datasets.
Limitations
- Based only on publicly available de-identified datasets, which may not reflect real-world clinical deployment complexity.
- Residual privacy risks may remain despite anonymization.
- Several downstream benchmarks are regularly sampled and only made irregular by synthetic random masking.
Experimental setup
Datasets: MIMIC-III MIMIC-IV PTB-XL Sleep-EDF WAVES CinC 2012 MIT-BIH JHU COVID-19 CDC-IHA CDC-Illness Heart Rate (Monash)
Benchmarks: CinC 2012 Heart Rate MIT-BIH CDC-IHA JH COVID-19 ILI SleepEDF PTB-XL MIMIC-III MIMIC-IV WAVES WHO FluNet (missing-rate study)
Metrics: RMSE MAE
Baselines: Time-MoE Moirai Moirai-MoE Moment TimeGPT Timer Lag-Llama TimesFM Chronos ContiFormer T-PatchGNN Neural-CDE
Builds on: Time-MoE RoPE Neural ODE Huber loss
Closest papers by meaning
- Knowledge-Empowered Dynamic Graph Network for Irregularly Sampled Medical Time Series (NeurIPS 2024 · similarity 0.85)
- Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHR Data (ICLR 2025 · similarity 0.85)
- JoLT: Jointly Learned Representations of Language and Time-Series for Clinical Time-Series Interpretation (Student Abstract) (AAAI 2024 · similarity 0.84)
- Risk Horizons: Structured Hypothesis Spaces for Longitudinal Clinical Prediction (NeurIPS 2026 · similarity 0.84)
- MOTOR: A Time-to-Event Foundation Model For Structured Medical Records (ICLR 2024 · similarity 0.84)
- Integrating Sequence and Image Modeling in Irregular Medical Time Series Through Self-Supervised Learning (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.