Research map / Time series / Foundation models for time series
Forging Time Series with Language: A Large Language Model Approach to Synthetic Data Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
SDForger converts univariate/multivariate time series into compact FastICA or FPC embedding tables, serializes them as text, fine-tunes an autoregressive LLM (GPT-2) and decodes sampled embeddings back into synthetic series.
Problem
Generating high-quality synthetic multivariate and univariate time series from few samples, where existing VAE, GAN and diffusion generators are trained from scratch per dataset and struggle with long dependencies and multivariate coupling.
Approach
Project time-series windows onto a small number of functional basis components (FastICA or FPC) to get a length-independent tabular embedding, then encode each row as fill-in-the-middle text with random feature-order permutation and fine-tune an autoregressive LLM to generate new embedding rows, decoded back to signals; text conditioning allows channel-specific generation.
Key results
- On multisample and univariate similarity, SDF-ICA$_3$ has the best normalized feature-based average (0.224 multisample, 0.149 univariate) and best average rank in univariate (2.429), while TimeVAE has the best multisample rank (2.143) and TimeVQVAE the best distance-based scores.
- In utility evaluation on bikesharing, SDF-ICA + OD achieves the best RMSE (0.487), MASE (0.801) and WQL (0.173) among combined-data settings.
- Generation time is substantially lower than all competitors, with TimeVAE the closest but over 4x slower, according to the paper's efficiency analysis.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ICA embeddings instead of FPC embeddings (ablation) | SDForger · Multisample | average rank: 4.714 -> 3.143 | Helped |
| use ICA embeddings instead of FPC embeddings (ablation) | SDForger · Univariate | average rank: 4.000 -> 2.429 | Helped |
| use ICA embeddings instead of FPC embeddings (ablation) | SDForger · Bikesharing | average rank: 3.22 -> 2.00 | Helped |
| use ICA embeddings instead of FPC embeddings (ablation) | SDForger · ETTH1 | average rank: 3.22 -> 2.00 | Helped |
| use ICA embeddings instead of FPC embeddings (ablation) | SDForger · Traffic | average rank: 3.22 -> 2.00 | Helped |
| train TTM on SDF-ICA synthetic data instead of original data | TTM · ETTH1 | RMSE: 0.658 -> 0.626 | Helped |
| train TTM on SDF-ICA synthetic data instead of original data | TTM · Bikesharing | RMSE: 0.495 -> 0.514 | Hurt |
| train TTM on SDF-ICA synthetic data instead of original data | TTM · Traffic | RMSE: 0.702 -> 0.655 | Helped |
| augment original data with SDF-ICA synthetic data | TTM · Bikesharing | RMSE: 0.495 -> 0.487 | Helped |
| augment original data with SDF-ICA synthetic data | TTM · ETTH1 | MASE: 1.820 -> 1.746 | Helped |
| augment original data with SDF-ICA synthetic data | TTM · Traffic | RMSE: 0.702 -> 0.750 | Hurt |
| increase the embedding dimension (ablation) | SDForger · Bikesharing count embeddings | missing-value rate: 3.87 -> 36.29 -> 58.82 | Hurt |
Surprising findings
- Traffic utility results show fine-tuning on real data is less effective than zero-shot and GAN-generated data, suggesting distribution shift between train and test.
- SDF-ICA ranks best on distance-based metrics yet TimeVQVAE, not SDForger, leads distance-based scores overall in the aggregated table.
Limitations
- Only autoregressive LLMs are supported; encoder-only models are future work.
- Full fine-tuning is used; high component counts relative to few instances cause unstable fine-tuning and higher discard rates.
- Performance on traffic and domains like tourism and finance is weaker, and the univariate and multivariate results are mixed against baselines.
Experimental setup
Datasets: Australian Electricity Appliances Bikesharing Carbon Capture Plant ETTH1 ECL Exchange Rate NN5 Tourism Traffic Traffic Monash Weather
Benchmarks: ETTH1 Bikesharing Traffic 12 public datasets (multisample/univariate similarity)
Metrics: MDD ACD SD KD ED DTW SHAP-RE RMSE MASE WQL
Baselines: TimeVAE TimeVQVAE RTSGAN SDEGAN LS4
Builds on: TimeVAE LLMTime Time-LLM TSGBench
Closest papers by meaning
- STEM-LTS: Integrating Semantic-Temporal Dynamics in LLM-driven Time Series Analysis (AAAI 2025 · similarity 0.85)
- Time-LLM: Time Series Forecasting by Reprogramming Large Language Models (ICLR 2024 · similarity 0.84)
- Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach (NeurIPS 2025 · similarity 0.84)
- AutoTimes: Autoregressive Time Series Forecasters via Large Language Models (NeurIPS 2024 · similarity 0.84)
- Kairos: Toward Adaptive and Parameter-Efficient Time Series Foundation Models (NeurIPS 2026 · similarity 0.83)
- Can LLMs Understand Time Series Anomalies? (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.