Research map / Time series / Foundation models for time series

Time-FFM: Towards LM-Empowered Federated Foundation Model for Time Series Forecasting

NeurIPS 2024 · Time series › Foundation models for time series · approach: Time series foundation models · 10 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Time-FFM builds a federated foundation model for time series forecasting by mapping time series to text-prototype tokens, adaptively selecting prompts from the LM's embeddings, and personalizing prediction heads over a frozen GPT-2 backbone.

Problem

Building foundation models for time series forecasting is hampered by data scarcity and privacy/copyright constraints on sharing cross-domain data, while existing LM-based time series models are centralized and need domain-specific parameters.

Approach

Combines cross-modality adaptation of a frozen GPT-2 (patch-based modality alignment plus attention-selected text-prototype prompts) with personalized federated training that shares encoder parameters globally and keeps prediction heads local.

Key results

• Time-FFM achieves average MSE 0.555 and MAE 0.434 over all forecasting windows, best among federated baselines (FedIT average 0.910) and better than UniTime (0.559 MSE), with 8 of 8 dataset-level best in TY1.
• In 5% few-shot forecasting, Time-FFM reports ~20% lower average MSE than UniTime (0.329 vs 0.411).
• In zero-shot forecasting (trained on ETTh1/ETTm1/ETTm2), Time-FFM average MSE is 0.310 vs 0.360 for the second-best FL method (FedIT).

Measured effects reported in the paper

Surprising findings

• Freezing the GPT-2 backbone performs comparably to full fine-tuning in the ablation, suggesting the modality alignment does most of the work.
• For all methods except UniTime, 10% few-shot results are worse than 5% few-shot results, which the authors attribute to frozen or partially tuned LMs retaining reasoning capability.

Limitations

• Training time is higher than the federated fine-tuning baselines (TY1).
• Performance is suboptimal in some cases compared with centralized methods.
• The modality alignment strategy and the correspondence between patch and word embeddings are not fully explored.

Experimental setup

Datasets: ETTh1 ETTh2 ETTm1 ETTm2 Electricity Weather Exchange ILI

Benchmarks: ETTh1 ETTh2 ETTm1 ETTm2 Electricity Weather Exchange ILI

Metrics: MSE MAE

Baselines: FedIT FedAdapter-H FedAdapter-P FedLoRA UniTime GPT4TS PatchTST TimesNet DLinear FEDformer Autoformer Informer

Builds on: UniTime GPT4TS PatchTST FedIT FedAdapter

Closest papers by meaning

• Federated Foundation Models on Heterogeneous Time Series (AAAI 2025 · similarity 0.90)
• Time-LLM: Time Series Forecasting by Reprogramming Large Language Models (ICLR 2024 · similarity 0.90)
• AutoTimes: Autoregressive Time Series Forecasters via Large Language Models (NeurIPS 2024 · similarity 0.88)
• $S^2$IP-LLM: Semantic Space Informed Prompt Learning with LLM for Time Series Forecasting (ICML 2024 · similarity 0.88)
• TimeCAP: Learning to Contextualize, Augment, and Predict Time Series Events with Large Language Model Agents (AAAI 2025 · similarity 0.87)
• TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignment (AAAI 2025 · similarity 0.87)