Research map / Time series / Foundation models for time series
MAESTRO : Adaptive Sparse Attention and Robust Learning for Multimodal Dynamic Time Series
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MAESTRO, a multimodal time-series framework combining SAX symbolic tokenization with a missingness symbol, adaptive sparse attention budgeting, sparse cross-modal attention over concatenated modality sequences, and a sparse MoE with modality-dropout curriculum.
Problem
Multimodal time-series learning from heterogeneous sensors where the primary modality is unclear, pairwise modality interactions are combinatorially expensive, and sensor failures cause arbitrary missing modalities.
Approach
Tokenize each modality with SAX (reserving a symbol for missingness), apply per-modality sparse attention with a learned budget conditioned on modality availability, concatenate modality tokens into one long sequence for sparse cross-modal attention, and route tokens through a sparse MoE trained with curriculum modality dropout.
Key results
- Under full observation, MAESTRO (with SAX) reaches 0.77 accuracy on WESAD, 0.83 on DaliaHAR, 0.88 on DSADS, and 0.78 on MIMIC-III, with average relative gains of about 4% over the best multimodal and 8% over the best multivariate baselines.
- Under 40% missing modalities, MAESTRO reports an average 9% improvement over baselines, e.g. 0.71 vs 0.59 accuracy for FlexMoE on WESAD.
- Replacing sparse with full attention in per-modal encoders improves WESAD accuracy by up to 3% while adding about 20% GFLOPs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add symbolic transformation (ablation) | MAESTRO · WESAD, full modalities | accuracy: 0.69 -> 0.78 | Helped |
| add symbolic transformation (ablation) | MAESTRO · WESAD, 40% missing | accuracy: 0.66 -> 0.71 | Helped |
| add modality embedding (ablation) | MAESTRO · WESAD, full modalities | accuracy: 0.70 -> 0.78 | Helped |
| add modality embedding (ablation) | MAESTRO · WESAD, 40% missing | accuracy: 0.55 -> 0.71 | Helped |
| add modality dropout (ablation) | MAESTRO · WESAD, full modalities | accuracy: 0.76 -> 0.78 | Helped |
| add modality dropout (ablation) | MAESTRO · WESAD, 40% missing | accuracy: 0.62 -> 0.71 | Helped |
| add adaptive attention budget (ablation) | MAESTRO · WESAD, full modalities | accuracy: 0.71 -> 0.78 | Helped |
| add adaptive attention budget (ablation) | MAESTRO · WESAD, 40% missing | accuracy: 0.67 -> 0.71 | Helped |
| add cross-modal SparseMoE (ablation) | MAESTRO · WESAD, full modalities | accuracy: 0.70 -> 0.78 | Helped |
| add cross-modal SparseMoE (ablation) | MAESTRO · WESAD, 40% missing | accuracy: 0.50 -> 0.71 | Helped |
| replace per-modal sparse attention with dense attention (ablation) | MAESTRO · WESAD | accuracy: $0.77_{\pm0.04}$ -> $0.80_{\pm0.03}$ | Helped |
| replace cross-modal sparse attention with dense attention (ablation) | MAESTRO · WESAD | accuracy: $0.77_{\pm0.04}$ -> $0.77_{\pm0.07}$ | No clear effect |
Surprising findings
- Full dense attention in the per-modal encoder gave higher WESAD accuracy (0.80) than the sparse version (0.77), at higher compute.
- On MIMIC-III, the gains are small and macro-F1 stays around 0.30, with the 'without SAX' variant performing comparably.
Limitations
- Only complete modality missingness is handled; asynchronous or irregular sampling is left to future work.
- Noise and asynchrony results are preliminary pilot experiments.
- MIMIC-III results show small gains and low absolute macro-F1.
Experimental setup
Datasets: WESAD DaLiA DSADS MIMIC-III
Benchmarks: WESAD DaliaHAR DSADS MIMIC-III
Metrics: accuracy macro-F1
Baselines: InceptionTime ResNet1D Transformer iTransformer LRTF Ensemble FuseMoE FlexMoE MULT ShaSpec
Builds on: SAX ProbSparse self-attention Sparse MoE MULT FlexMoE
Closest papers by meaning
- Empowering Time Series Analysis with Large-Scale Multimodal Pretraining (NeurIPS 2026 · similarity 0.87)
- Time-MMD: Multi-Domain Multimodal Dataset for Time Series Analysis (NeurIPS 2024 · similarity 0.85)
- TRACE: Grounding Time Series in Context for Multimodal Embedding and Retrieval (NeurIPS 2025 · similarity 0.85)
- Large Pre-trained time series models for cross-domain Time series analysis tasks (NeurIPS 2024 · similarity 0.85)
- ITFormer: Bridging Time Series and Natural Language for Multi-Modal QA with Large-Scale Multitask Dataset (ICML 2025 · similarity 0.85)
- Revitalizing Medical Time Series with Vision-Informed Retrieval: A Vision-Language Perspective (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.