Research map / Time series / Causal and generative time series
SDformer: Similarity-driven Discrete Transformer For Time Series Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SDformer, a two-stage discrete token model for time series generation using a similarity-driven VQ-VAE tokenizer plus autoregressive (SDformer-ar) or masked (SDformer-m) Transformers over the tokens.
Problem
Diffusion-based time series generators are slow at inference and their generated sample quality can still be improved.
Approach
Replace distance-based codebook lookup in VQ-VAE with similarity-based (inner product on unit-normalized vectors) quantization, then model the resulting discrete time series tokens with an autoregressive or masked Transformer, using random replacement as augmentation for the AR variant.
Key results
- SDformer-ar achieves the best discriminative score on most datasets, e.g. 0.006 on Energy vs. 0.122 for Diffusion-TS and 0.017 on fMRI vs. 0.167.
- Similarity-driven quantization reduces discriminative score relative to distance-driven quantization by 35.6% for SDformer-ar and 46.2% for SDformer-m (as reported by the authors).
- Inference time is about 2.9 s versus 27-85 s for Diffusion-TS across datasets, per the authors' measurements.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SDformer-ar instead of Diffusion-TS | time-series generation · Stocks | Discriminative Score: 0.067±.015 -> 0.010±.006 | Helped |
| use SDformer-ar instead of Diffusion-TS | time-series generation · ETTh | Context-FID Score: 0.116±.010 -> 0.008±.001 | Helped |
| use SDformer-ar instead of Diffusion-TS | time-series generation · Sines | Discriminative Score: 0.006±.007 -> 0.006±.004 | No clear effect |
| replace similarity-driven quantization with distance-driven quantization (ablation) | SDformer-ar · ETTh | Discriminative Score: 0.003±.001 -> 0.010±.005 | Hurt |
| replace similarity-driven quantization with distance-driven quantization (ablation) | SDformer-m · Stocks | Context-FID Score: 0.034±.008 -> 0.123±.009 | Hurt |
| replace discrete tokens with continuous tokens (ablation) | SDformer-ar · ETTh | Discriminative Score: 0.003±.001 -> 0.145±.020 | Hurt |
| use continuous tokens and provide the actual first token at inference (ablation) | SDformer-ar · Energy | Discriminative Score: 0.006±.004 -> 0.277±.007 | Hurt |
| replace Transformer with LSTM (ablation) | SDformer-ar · fMRI | Context-FID Score: 0.015±.001 -> 0.018±.001 | Hurt |
| replace Transformer with LSTM (ablation) | SDformer-ar · Stocks | Discriminative Score: 0.010±.006 -> 0.008±.005 | Helped |
| increase model size from 1.4M to 44.9M (ablation) | SDformer · Energy | Discriminative Score: 0.149±.007 -> 0.006±.004 | Helped |
| use SDformer-s instead of Diffusion-TS | time-series generation · Stocks | Inference Time (s): 30.01 -> 2.59 | Helped |
| use SDformer-ar instead of Diffusion-TS | time-series generation · MuJoCo | Discriminative Score: 0.008±.002 -> 0.008±.005 | No clear effect |
Surprising findings
- SDformer-ar substantially outperforms SDformer-m, contrary to findings in the visual domain where masked token modeling is strong.
Limitations
- Strong generation requires a large codebook and more parameters, increasing memory pressure.
- Evaluation limited to six datasets with standard generation metrics.
Experimental setup
Datasets: Sines Stocks ETTh MuJoCo Energy fMRI
Benchmarks: Sines Stocks ETTh MuJoCo Energy fMRI
Metrics: Discriminative Score Predictive Score Context-FID Score Inference time MSE for forecasting
Baselines: Diffusion-TS TimeGAN TimeVAE Diffwave DiffTime Cot-GAN
Builds on: VQ-VAE MaskGIT Diffusion-TS T2M-GPT
Closest papers by meaning
- Diffusion-TS: Interpretable Diffusion for General Time Series Generation (ICLR 2024 · similarity 0.88)
- HDT: Hierarchical Discrete Transformer for Multivariate Time Series Forecasting (AAAI 2025 · similarity 0.88)
- Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach (NeurIPS 2025 · similarity 0.87)
- Probabilistic Time Series Modeling with Decomposable Denoising Diffusion Model (ICML 2024 · similarity 0.87)
- Utilizing Image Transforms and Diffusion Models for Generative Modeling of Short and Long Time Series (NeurIPS 2024 · similarity 0.87)
- TimeDART: A Diffusion Autoregressive Transformer for Self-Supervised Time Series Representation (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.