Research map / Time series / Foundation models for time series
IMTS is Worth Time $\times$ Channel Patches: Visual Masked Autoencoders for Irregular Multivariate Time Series Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adapts a visual masked autoencoder (MAE, ViT-base) to irregular multivariate time series forecasting by patchifying IMTS into time×channel patches, complementing them with GCN cross-channel information, and using a coarse-to-fine patch2point decoder with two-stage SSL and fine-tuning.
Problem
Forecasting irregular multivariate time series (IMTS), where unaligned multi-channel signals and extensive missing values make reliable temporal pattern capture difficult, and existing foundation models target only regularly sampled series.
Approach
Treat IMTS as a time×channel image-like structure: TTCN-based intra-section patch extraction with mask indicators, GCN-based cross-channel complementation of missing patches, a pre-trained visual MAE for time-wise patch reconstruction, and a coarse-to-fine patch2point decoder, trained with IMTS-specific self-supervised reconstruction before supervised fine-tuning.
Key results
- VIMTS (100% data) achieves MSE 4.81e-3 / MAE 3.54e-2 on PhysioNet, versus 4.98e-3 for t-PatchGNN, and the lowest MSE/MAE on all four datasets per the paper.
- VIMTS with only 20% of training data matches or approaches t-PatchGNN with full data (e.g., PhysioNet MSE 4.93e-3 vs 4.98e-3 for t-PatchGNN).
- VisionTS (zero/linear interpolation) performs far worse on IMTS, e.g., PhysioNet MSE 42.41e-3 and 40.50e-3.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use VIMTS with visual MAE pretraining (ablation) | VIMTS · PhysioNet | MSE×10^-3: 5.13 ± 0.04 -> 4.81 ± 0.07 | Helped |
| use IMTS-specific self-supervised training (ablation) | VIMTS · PhysioNet | MSE×10^-3: 5.46 ± 0.30 -> 4.81 ± 0.07 | Helped |
| combine visual pretraining with IMTS-specific self-supervised training (ablation) | VIMTS · PhysioNet | MSE×10^-3: 5.70 ± 0.42 -> 4.81 ± 0.07 | Helped |
| add cross-channel GCN (ablation) | VIMTS · MIMIC | MSE×10^-2: 2.25 ± 0.02 -> 1.36 ± 0.02 | Helped |
| add cross-channel GCN (ablation) | VIMTS · PhysioNet | MSE×10^-3: 4.94 ± 0.03 -> 4.81 ± 0.07 | No clear effect |
| use a visual MAE rather than a vanilla Transformer encoder (ablation) | VIMTS · PhysioNet | MSE×10^-3: 5.57 ± 0.34 -> 4.81 ± 0.07 | Helped |
| use Patch2Point in both SSL and fine-tuning rather than direct projection in both stages (ablation) | VIMTS · PhysioNet | MSE×10^-3: 4.98±0.07 -> 4.81 ± 0.07 | Helped |
| use Patch2Point in fine-tuning instead of direct projection (ablation) | VIMTS with Patch2Point in SSL · PhysioNet | MSE×10^-3: 4.97 ± 0.07 -> 4.81 ± 0.07 | Helped |
| fine-tune only normalization layers rather than all parameters (ablation) | VIMTS · PhysioNet | MSE×10^-3: 4.95 ± 0.06 -> 4.81 ± 0.07 | Helped |
| fine-tune only normalization layers rather than attention layers (ablation) | VIMTS · PhysioNet | MSE×10^-3: 4.91 ± 0.04 -> 4.81 ± 0.07 | Helped |
| fine-tune only normalization layers rather than bias terms (ablation) | VIMTS · PhysioNet | MSE×10^-3: 4.86 ± 0.05 -> 4.81 ± 0.07 | Helped |
| fine-tune only normalization layers rather than MLP blocks (ablation) | VIMTS · PhysioNet | MSE×10^-3: 4.91 ± 0.03 -> 4.81 ± 0.07 | Helped |
Surprising findings
- Direct VisionTS-style adaptation to IMTS performs much worse than even the simple baselines, indicating that naive visual foundation model use fails on irregular sampling.
- The authors could not reproduce t-PatchGNN's MIMIC results due to preprocessing issues and re-ran it with corrected preprocessing.
Limitations
- Fixed patch sizes and number of channels limit structural flexibility and zero-shot transfer.
- Scaling the MAE beyond base size did not improve results and caused memory issues.
- Model has about 111M trainable parameters and relatively high training cost (87.8 s/epoch).
Experimental setup
Datasets: PhysioNet Human Activity USHCN MIMIC
Benchmarks: PhysioNet Human Activity USHCN MIMIC
Metrics: MSE MAE
Baselines: t-PatchGNN VisionTS ViTST CRU Warpformer PatchTST TimesNet GRU-D
Builds on: VisionTS t-PatchGNN Masked Autoencoder
Closest papers by meaning
- TimeCHEAT: A Channel Harmony Strategy for Irregularly Sampled Multivariate Time Series Analysis (AAAI 2025 · similarity 0.90)
- VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasters (ICML 2025 · similarity 0.88)
- OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain (NeurIPS 2025 · similarity 0.87)
- Irregular Multivariate Time Series Forecasting: A Transformable Patching Graph Neural Networks Approach (ICML 2024 · similarity 0.86)
- Learning to Embed Time Series Patches Independently (ICLR 2024 · similarity 0.86)
- TSLANet: Rethinking Transformers for Time Series Representation Learning (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.