Research map / Time series / Long-term forecasting
STEM-LTS: Integrating Semantic-Temporal Dynamics in LLM-driven Time Series Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes STEM-LTS, an LLM-based time series forecaster that decomposes series into trend/season/residual, aligns components with learnable prompts and CLIP embeddings, and balances losses dynamically, fine-tuned from TEMPO.
Problem
Long-term time series forecasting with LLMs struggles with multi-scale dynamics, seasonal patterns, and explicit alignment between numerical time series patterns and semantic representations.
Approach
Combines time series decomposition (trend, seasonal, residual) with per-component encoders, prompt-based contrastive and CLIP-based semantic alignment, a covariance-based decomposition independence loss, and a log-sum-exp dynamic multi-task loss on top of the TEMPO LLM backbone.
Key results
- On ECL at horizon 96, STEM-LTS achieves 0.142 MSE / 0.251 MAE vs. 0.196/0.295 for TEMPO (as reported in Table 1).
- On EBITDA (outliers removed at 0.8), STEM-LTS obtains the lowest SMAPE in all sectors, e.g., 30.35 for Consumer Cyclical vs. 33.42 for T5 (second best).
- STEM-LTS is reported to consistently improve temporal component correlation during fine-tuning while TEMPO fluctuates (Figure 3).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Integrate time-series decomposition, semantic-temporal alignment, and adaptive multi-task learning in STEM-LTS | TEMPO backbone · ECL, Traffic, Weather, Ettm1, Ettm2, Etth1, Etth2; horizon 96 | MSE: 0.142 vs 0.151; 0.428 vs 0.507; 0.161 vs 0.204; 0.316 vs 0.468; 0.138 vs 0.176; 0.336 vs 0.381; 0.256 vs 0.307 | Helped |
| Integrate STEM-LTS components | TEMPO backbone · EBITDA, four sectors | SMAPE: 30.35/31.56 vs 33.42/35.33; 25.84/26.50 vs 26.44/26.79; 26.99/27.38 vs 27.30/28.12; 28.80/29.31 vs 30.10/30.64 | Helped |
| Integrate STEM-LTS components | TEMPO backbone · GDELT events 11, 17, and 19 | SMAPE: 39.88 vs 41.75; 41.59 vs 42.56; 44.00 vs 45.13 | Helped |
Limitations
- Theoretical analyses are mentioned in contributions but not presented in the main text.
- Hyperparameters were tuned by grid search and cross-validation with limited reported detail; results compared mainly against TEMPO-derived and older baselines.
- Several benchmark tables show STEM-LTS not best across all settings in the reported rows (e.g., Traffic at some horizons).
Experimental setup
Datasets: EBITDA GDELT ECL Traffic Weather ETTm1 ETTm2 ETTh1 ETTh2
Benchmarks: ECL Traffic Weather ETTm1 ETTm2 ETTh1 ETTh2 EBITDA GDELT
Metrics: MSE MAE SMAPE
Baselines: TEMPO LLM4TS GPT4TS T5 PatchTST TimesNet FEDformer ETSformer Informer DLinear
Builds on: TEMPO CLIP GPT-2 PatchTST
Closest papers by meaning
- Time-LLM: Time Series Forecasting by Reprogramming Large Language Models (ICLR 2024 · similarity 0.91)
- TimeXL: Explainable Multi-modal Time Series Prediction with LLM-in-the-Loop (NeurIPS 2025 · similarity 0.90)
- $S^2$IP-LLM: Semantic Space Informed Prompt Learning with LLM for Time Series Forecasting (ICML 2024 · similarity 0.89)
- AutoTimes: Autoregressive Time Series Forecasters via Large Language Models (NeurIPS 2024 · similarity 0.89)
- TimeCAP: Learning to Contextualize, Augment, and Predict Time Series Events with Large Language Model Agents (AAAI 2025 · similarity 0.89)
- TimeBase: The Power of Minimalism in Efficient Long-term Time Series Forecasting (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.