Research map / Time series / Foundation models for time series
Federated Foundation Models on Heterogeneous Time Series
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FFTS, a federated learning approach that pretrains a time series foundation model from scratch by treating each dataset/domain as a client with a local Transformer, using an adaptive trend-awareness MoE module, uniform masking, and heterogeneous knowledge alignment regularization.
Problem
Training a general-purpose time series foundation model from scratch across heterogeneous domains is hard because cross-domain fusion of time series yields statistical heterogeneity, and centralized pooling raises privacy concerns.
Approach
Each data-holding organization acts as an FL client training a local encoder-only Transformer; the server aggregates them into a global TSFM. Adds an adaptive trend-awareness module (four timescale experts with a gating network), uniform masked reconstruction for local training, and a regularizer aligning ATM parameters between local and global models.
Key results
- Long-term forecasting: FFTS (FT) averages 0.473 MSE, about 9.52% better than Time-LLM on six datasets; vanilla FedAvg is within 1.7% of Time-LLM.
- Anomaly detection: FFTS averages 88.10 F1 versus 86.72 for GPT4TS (+1.02%).
- Imputation: FFTS averages 0.034 MSE, reducing MSE by 14.7% relative to GPT4TS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use federated FFTS instead of centralized FFTS-Cen | FFTS · Long-term forecasting (Avg.) | MSE: 0.475 -> 0.473 | Helped |
| Use federated FFTS instead of centralized FFTS-Cen | FFTS · Imputation (Avg.) | MSE: 0.035 -> 0.034 | Helped |
| Use federated FFTS instead of centralized FFTS-Cen | FFTS · Anomaly detection (Avg.) | F1: 87.69 -> 88.10 | Helped |
| Set ATM top-k to 3 rather than 1 or 2 (ablation) | FFTS · Federated pretraining, L_m=16 | MSE: .432 -> .421; .427 -> .421 | Helped |
| Set ATM top-k to 4 rather than 3 (ablation) | FFTS · Federated pretraining, L_m=16 | MSE: .421 -> .445 | Hurt |
| Increase client participation rate from 10% to 50% (ablation) | FFTS · Federated pretraining, L_m=16 | MSE: .500 -> .421 | Helped |
| Set regularization weight lambda to 5e^{-2} instead of the default 1e^{-1} (ablation) | FFTS · Federated pretraining, L_m=16 | MSE: .421 -> .430 | Hurt |
| Set regularization weight lambda to 2e^{-1} instead of the default 1e^{-1} (ablation) | FFTS · Federated pretraining, L_m=16 | MSE: .421 -> .437 | Hurt |
| Add the ATM and heterogeneous knowledge alignment (ablation) | FFTS · Federated pretraining (Avg.) | MSE: .444 -> .414 | Helped |
| Add heterogeneous knowledge alignment (ablation) | FFTS · Federated pretraining (Avg.) | MSE: .431 -> .414 | Helped |
| Add the day-aware ATM layer (ablation) | FFTS · Federated pretraining, L_m=16 | MSE: .427 -> .421 | Helped |
Surprising findings
- Heterogeneous FL algorithms (FedProx, pFedMe) underperform vanilla FedAvg on pretraining in this setting, contrary to their expected benefits on natural image tasks.
Limitations
- Efficiency and privacy claims are analytical and not empirically measured.
- Zero-shot evaluation limited to the ETT dataset.
- Pretraining uses 18 datasets with a fixed uniform look-back length of 512.
Experimental setup
Datasets: 18 pretraining time series datasets (listed in appendix) ETTh1 ETTh2 ETTm1 ETTm2 Weather ILI M4 SMD MSL SMAP SWaT
Benchmarks: ETT Weather ILI M4 SMD MSL SMAP SWaT PSM
Metrics: MSE MASE SMAPE OWA F1-score
Baselines: FedAvg FedProx pFedMe FFTS-Cen Time-LLM GPT4TS PatchTST TimesNet DLinear LLMTime
Builds on: FedAvg FedProx pFedMe GPT4TS PatchTST
Closest papers by meaning
- Time-FFM: Towards LM-Empowered Federated Foundation Model for Time Series Forecasting (NeurIPS 2024 · similarity 0.90)
- FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learning (AAAI 2024 · similarity 0.88)
- FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning (NeurIPS 2025 · similarity 0.87)
- One-Shot Heterogeneous Federated Learning with Local Model-Guided Diffusion Models (ICML 2025 · similarity 0.86)
- An Aggregation-Free Federated Learning for Tackling Data Heterogeneity (CVPR 2024 · similarity 0.86)
- Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.