Research map / Time series / Foundation models for time series
Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Benchmarks time series generators under data scarcity and proposes a unified, pre-trained diffusion model (ImagenTime-based) with DyConv channel adaptation and dataset tokens that fine-tunes well on few samples.
Problem
Time series generative models degrade substantially when training data is scarce (few-shot), and existing models are trained separately per dataset.
Approach
Pre-train one image-based diffusion model (ImagenTime backbone with delay embedding) across ~300k heterogeneous series, using a DyConv layer that bicubically interpolates a canonical kernel to handle varying channel counts, and a learnable dataset token for domain conditioning; then fine-tune with a new token on small target subsets.
Key results
- Fine-tuned model on 5% of data outperforms ImagenTime trained on full data and gives average gains of 55.72% (Disc.) and 54.25% (contextFID) over competing models.
- On 10-sample setting, Disc. score 0.259 vs 0.357 for ImagenTime (26.61% improvement), c-FID 3.800 vs 5.393.
- DyConv vs padded baseline: averaged Disc. 0.110 vs 0.291 at 5% subset; DyConv uses ~0.18 GB vs 2.01 GB memory for padding.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use unified pre-training and fine-tuning instead of training ImagenTime from scratch | Time-series generation · 5% data | Discriminative Score: 0.110 vs 0.321 | Helped |
| Use unified pre-training and fine-tuning instead of training ImagenTime from scratch | Time-series generation · 5% data | contextFID: 0.674 vs 3.464 | Helped |
| Use unified pre-training and fine-tuning instead of training ImagenTime from scratch | Time-series generation · #10 samples | Predictive Score: 0.489 vs 0.467 | Hurt |
| Replace static channel padding with DyConv (ablation) | Unified model · Few-shot datasets, averaged across evaluation datasets | Discriminative Score: 0.291 vs 0.110; 0.250 vs 0.083; 0.240 vs 0.066; 0.381 vs 0.259; 0.372 vs 0.190; 0.355 vs 0.149 | Helped |
| Replace static channel padding with DyConv (ablation) | Unified model · Few-shot datasets, averaged across evaluation datasets | contextFID: 4.190 vs 0.674; 4.864 vs 0.578; 3.532 vs 1.086; 7.889 vs 3.800; 5.940 vs 1.582; 5.905 vs 0.987 | Helped |
| Add dataset-token conditioning during pre-training (ablation) | Unified model, without fine-tuning on pre-training datasets · Pre-training datasets | Discriminative Score: 0.252 vs 0.193 | Helped |
| Add dataset-token conditioning during pre-training (ablation) | Unified model, without fine-tuning on pre-training datasets · Pre-training datasets | contextFID: 11.07 vs 4.84 | Helped |
| Remove dataset-token conditioning (ablation) | Unified model fine-tuned on few-shot benchmark datasets · Few-shot benchmark, #10 samples | contextFID: 3.80 vs 2.42 | Helped |
| Remove dataset-token conditioning (ablation) | Unified model fine-tuned on few-shot benchmark datasets · Few-shot benchmark, #25 samples | contextFID: 1.58 vs 1.23 | Helped |
| Increase pre-training model size from Large to XL (ablation) | Fine-tuned unified model · 64-step time-series generation | Discriminative Score: 0.13 to 0.13 | Hurt |
| Increase pre-training model size from Large to XL (ablation) | Fine-tuned unified model · 64-step time-series generation | contextFID: 4.56 to 4.51 | Helped |
| Add pre-training before fine-tuning (ablation) | Base model · 64-step few-shot generation | contextFID: 15.87 to 4.85 | Helped |
Surprising findings
- Pre-training without dataset tokens performs comparably after fine-tuning, but the non-fine-tuned unconditional model samples poorly (Disc. 0.252, c-FID 11.07 vs 0.193 and 4.84 with tokens).
- Larger XL model (40M) slightly underperforms Large (26M) on discriminative score after fine-tuning, suggesting diminishing returns.
Limitations
- Data augmentation strategies were deferred to future work and not investigated.
- Some datasets (Weather, ECG200) remain challenging under limited data.
- LoRA and bias-only fine-tuning gave poor results compared with full fine-tuning.
Experimental setup
Datasets: Stocks Energy ETTh1 Exchange MSL SMAP PSM SMD ECG5000 NonInvasiveFetalECGThorax1 SelfRegulationSCP2 Blink
Benchmarks: MuJoCo ETTm1 ETTm2 ETTh2 Sine Weather ILI Saugeen River Flow ECG200 SelfRegulationSCP1 AirQuality StarLightCurves
Metrics: Discriminative Score Predictive Score contextFID
Baselines: ImagenTime DiffusionTS KoVAE TimeGAN
Builds on: ImagenTime EDM DyLinear AdaGN
Closest papers by meaning
- Utilizing Image Transforms and Diffusion Models for Generative Modeling of Short and Long Time Series (NeurIPS 2024 · similarity 0.93)
- Timer: Generative Pre-trained Transformers Are Large Time Series Models (ICML 2024 · similarity 0.91)
- TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting (ICLR 2024 · similarity 0.91)
- TSGM: A Flexible Framework for Generative Modeling of Synthetic Time Series (NeurIPS 2024 · similarity 0.90)
- TimeTok: Granularity-Controllable Time-Series Generation via Hierarchical Tokenization (NeurIPS 2026 · similarity 0.90)
- TimeDP: Learning to Generate Multi-Domain Time Series with Domain Prompts (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.