Research map / Time series / Foundation models for time series
TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TimeCMA, a dual-branch LLM-empowered multivariate time series forecasting framework that aligns a time series encoder with frozen GPT-2 prompt embeddings via channel-wise similarity retrieval, using only the cached last-token embedding for efficiency.
Problem
Multivariate time series forecasting with LLMs suffers from data entanglement between time series and text embeddings, and high computational cost from long textual prompts that are reprocessed each epoch.
Approach
Use disentangled but weak time series embeddings to retrieve robust time series components from entangled but robust LLM prompt embeddings via cross-modality (channel-wise) attention; design prompts so the last token summarizes temporal information, and store those last-token embeddings offline so the frozen LLM is not rerun during training or inference.
Key results
- TimeCMA achieves best average MSE/MAE on all eight datasets in the main table, e.g., ETTh1 avg MSE 0.423 / MAE 0.431 versus Time-LLM 0.448 / 0.443.
- Versus OFA, TimeCMA reports an average improvement of 16.1% in MSE and 11.9% in MAE.
- Efficiency: on ETTm1-96 TimeCMA has 17.99M trainable parameters, 821 MiB memory and 0.09 s/iter versus Time-LLM at 44.66M, 28,882 MiB and 1.08 s/iter.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace cross-modality alignment with concatenation (ablation) | TimeCMA · Eight forecasting datasets | MSE | Hurt |
| Remove the LLM-empowered branch (ablation) | TimeCMA · Eight forecasting datasets | MSE | Hurt |
| Remove the time series encoder (ablation) | TimeCMA · Eight forecasting datasets | MSE | Hurt |
| Remove the prompt encoder (ablation) | TimeCMA · Eight forecasting datasets | MSE | No clear effect |
| Remove the multivariate Transformer decoder (ablation) | TimeCMA · Eight forecasting datasets | MSE | Hurt |
| Use Prompt 5, which summarizes time-series trends and ends in a numerical value (ablation) | TimeCMA prompt design · ETTm1 and ETTm2 forecasting | MSE | Helped |
| Use a prompt with a numerical last token (ablation) | TimeCMA prompt design · ETTm1 and ETTm2 forecasting | MSE | Helped |
| Use Prompt 3, which averages the time series (ablation) | TimeCMA prompt design · ETTm1 and ETTm2 forecasting | MSE | Mixed |
| Use Prompt 2, which emphasizes historical time information (ablation) | TimeCMA prompt design · ETTm1 and ETTm2 forecasting | MSE | Mixed |
| Use TimeCMA instead of OFA | Time series forecasting · Eight datasets | MSE: +16.1% | Helped |
| Use TimeCMA instead of UniTime | Time series forecasting · Eight datasets | MSE: about 13.9% | Helped |
Surprising findings
- Time-series-only models with no LLM (e.g., PatchTST, TimesNet) are competitive on some datasets, and TimesNet beats TimeCMA on ILI MAE at 24 steps and on Weather 720-step MSE.
Limitations
- Relies on a single frozen GPT-2 backbone; effects of larger LLMs are not studied in the text.
- Experiments are limited to eight datasets and seven baselines, with the test batch size set to 1 for fairness.
Experimental setup
Datasets: ETTm1 ETTm2 ETTh1 ETTh2 ECL FRED-MD ILI Weather
Benchmarks: ETTm1 ETTm2 ETTh1 ETTh2 ECL FRED-MD ILI Weather
Metrics: MSE MAE
Baselines: Time-LLM UniTime OFA iTransformer PatchTST TimesNet DLinear
Builds on: Time-LLM iTransformer PatchTST GPT-2
Closest papers by meaning
- $S^2$IP-LLM: Semantic Space Informed Prompt Learning with LLM for Time Series Forecasting (ICML 2024 · similarity 0.93)
- CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuning (AAAI 2025 · similarity 0.92)
- Time-LLM: Time Series Forecasting by Reprogramming Large Language Models (ICLR 2024 · similarity 0.91)
- GPT4MTS: Prompt-based Large Language Model for Multimodal Time-series Forecasting (AAAI 2024 · similarity 0.90)
- ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data (AAAI 2025 · similarity 0.89)
- AutoTimes: Autoregressive Time Series Forecasters via Large Language Models (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.