Research map / Time series / Long-term forecasting
Multi-Modal View Enhanced Large Vision Models for Long-Term Time Series Forecasting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DMMV, a decomposition-based multi-modal view framework that pairs a MAE-based large vision model (visual view of imaged time series) with a linear/Transformer numerical forecaster, using trend-seasonal decomposition (DMMV-s) and a backcast-residual adaptive decomposition (DMMV-a) for long-term time series forecasting.
Problem
Long-term time series forecasting using multi-modal views (numerical and imaged representations) with pre-trained large vision models, where existing LVM forecasters are biased toward periodicity and existing multi-view fusion (Time-VLM) is suboptimal.
Approach
Decompose the series into seasonal and trend parts so the MAE-based visual forecaster models periodic components and a numerical linear/Transformer forecaster models trend; the adaptive variant learns the split via a backcast-residual mechanism (BCmask) that masks left/right halves of the periodic image and feeds the residual to the numerical forecaster, fused with a learned gate.
Key results
- DMMV-a achieves the best MSE on 6 of 8 benchmark datasets and 43 first-place results (MSE/MAE) in the main table, versus 9 for VisionTS and 9 for Time-VLM.
- On ETTh1 average, DMMV-a reaches MSE 0.395 / MAE 0.414, vs. VisionTS 0.407 / 0.419 and Time-VLM 0.405 / 0.420.
- Ablation: removing the backcast-residual decomposition raises Illness MSE from 1.407 to 1.712 (a 21.68% degradation).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Dmmv-a instead of the strongest listed baseline | long-term time-series forecasting · ETTh1 | MSE (Avg.): 0.405 -> 0.395 | Helped |
| Use Dmmv-a instead of the strongest listed baseline | long-term time-series forecasting · ETTh2 | MSE (Avg.): 0.330 -> 0.337 | Hurt |
| Use Dmmv-a instead of the strongest listed baseline | long-term time-series forecasting · ETTm1 | MSE (Avg.): 0.351 -> 0.340 | Helped |
| Use Dmmv-a instead of the strongest listed baseline | long-term time-series forecasting · ETTm2 | MSE (Avg.): 0.248 -> 0.256 | Hurt |
| Use Dmmv-a instead of the strongest listed baseline | long-term time-series forecasting · Illness | MSE (Avg.): 1.443 -> 1.407 | Helped |
| Use Dmmv-a instead of the strongest listed baseline | long-term time-series forecasting · Electricity | MSE (Avg.): 0.158 -> 0.158 | No clear effect |
| Use Dmmv-a instead of the strongest listed baseline | long-term time-series forecasting · Weather | MSE (Avg.): 0.225 -> 0.217 | Helped |
| Use Dmmv-a instead of the strongest listed baseline | long-term time-series forecasting · Traffic | MSE (Avg.): 0.386 -> 0.382 | Helped |
| replace the linear numerical forecaster with a Transformer (ablation) | Dmmv-a · ETTh1, ETTm1, Illness, Weather | MSE (Avg.): ETTh1 0.395 -> 0.407; ETTm1 0.340 -> 0.339; Illness 1.407 -> 1.442; Weather 0.217 -> 0.219 | Mixed |
| replace MAE with SimMIM (ablation) | Dmmv-a · ETTh1, ETTm1, Illness, Weather | MSE (Avg.): ETTh1 0.395 -> 0.407; ETTm1 0.340 -> 0.345; Illness 1.407 -> 1.649; Weather 0.217 -> 0.227 | Hurt |
| replace gate fusion with summation (ablation) | Dmmv-a · ETTh1, ETTm1, Illness, Weather | MSE (Avg.): ETTh1 0.395 -> 0.414; ETTm1 0.340 -> 0.352; Illness 1.407 -> 1.606; Weather 0.217 -> 0.233 | Hurt |
| remove BCmask (ablation) | Dmmv-a · ETTh1, ETTm1, Illness, Weather | MSE (Avg.): ETTh1 0.395 -> 0.426; ETTm1 0.340 -> 0.349; Illness 1.407 -> 1.493; Weather 0.217 -> 0.221 | Hurt |
Surprising findings
- Fine-tuning only the normalization layers of the LVM outperforms freezing it entirely (e.g., Weather MSE 0.217 vs 0.246).
- The gate weights place more weight on the visual forecaster, yet the numerical forecaster remains essential since DMMV-a outperforms VisionTS alone.
Limitations
- Sensitivity to imaging segment length (period) inherited from the LVM forecaster; DMMV-a is not consistently more robust across datasets.
- The visual backbone is reused three times per forward pass under BCmask, adding computational overhead.
- Time series-to-image resizing to 224x224 may distort temporal structure.
Experimental setup
Datasets: ETTh1 ETTh2 ETTm1 ETTm2 Weather Illness Traffic Electricity ImageNet-1K
Benchmarks: ETTh1 ETTh2 ETTm1 ETTm2 Weather Illness Traffic Electricity
Metrics: MSE MAE
Baselines: Time-VLM VisionTS Time-LLM GPT4TS CALF PatchTST FEDformer Autoformer Stationary ETSformer Informer DLinear
Builds on: VisionTS MAE PatchTST N-BEATS Time-VLM
Closest papers by meaning
- Empowering Time Series Analysis with Large-Scale Multimodal Pretraining (NeurIPS 2026 · similarity 0.87)
- Time-VLM: Exploring Multimodal Vision-Language Models for Augmented Time Series Forecasting (ICML 2025 · similarity 0.87)
- VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasters (ICML 2025 · similarity 0.86)
- HDT: Hierarchical Discrete Transformer for Multivariate Time Series Forecasting (AAAI 2025 · similarity 0.86)
- CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuning (AAAI 2025 · similarity 0.86)
- Time-LLM: Time Series Forecasting by Reprogramming Large Language Models (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.