Research map / Time series / Causal and generative time series
Latent Diffusion Transformer for Probabilistic Time Series Forecasting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Latent Diffusion Transformer (LDT), a two-stage model with a statistics-aware autoencoder and a non-autoregressive diffusion transformer with self-conditioning guidance, for high-dimensional probabilistic multivariate time series forecasting.
Problem
Probabilistic forecasting of high-dimensional multivariate time series is hard: autoregressive diffusion models accumulate errors, are slow, and struggle to scale to high dimensions, and non-stationary statistics hamper latent autoencoders.
Approach
Compress multivariate series into a latent space with a symmetric autoencoder using adaptive variance-updating normalization, then run a non-autoregressive transformer latent diffusion model (x-prediction) with a self-conditioning and classifier-free-style guidance mechanism.
Key results
- LDT reduces CRPS-sum on Electricity from 0.025 to 0.021 (16%) and on Traffic from 0.047 to 0.040 (14.8%) versus the best prior generative baselines.
- On Solar, LDT achieves CRPS-sum 0.253 versus 0.275 for SSSD, the next-best reported baseline.
- Predicting x0 rather than noise epsilon substantially improves CRPS-sum across all five datasets (e.g., Solar 0.528 vs 0.253).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LDT rather than the strongest listed baseline | LDT · Solar | CRPS-sum: 0.124(.014) -> 0.253(.002) | Hurt |
| use LDT rather than the strongest listed baseline | LDT · Solar | MSE: 5.4e2 -> 7.7e2 | Hurt |
| use LDT rather than the strongest listed baseline | LDT · Electricity | CRPS-sum: 0.022(.003) -> 0.021(.001) | Helped |
| use LDT rather than the strongest listed baseline | LDT · Electricity | MSE: 1.8e5 -> 1.6e5 | Helped |
| use LDT rather than the strongest listed baseline | LDT · Traffic | CRPS-sum: 0.047(.002) -> 0.040(.000) | Helped |
| use LDT rather than the strongest listed baseline | LDT · Traffic | MSE: 4.4e-4 -> 4.1e-4 | Helped |
| use LDT rather than the strongest listed baseline | LDT · Taxi | CRPS-sum: 0.130(.011) -> 0.125(.007) | Helped |
| use LDT rather than the strongest listed baseline | LDT · Taxi | MSE: 2.4e -> 2.2e | Helped |
| use LDT rather than the strongest listed baseline | LDT · Wikipedia | CRPS-sum: 0.067(.002) -> 0.061(.002) | Helped |
| use LDT rather than the strongest listed baseline | LDT · Wikipedia | MSE: 3.0e7 -> 2.92e7 | Helped |
| add self-conditioning (ablation) | LDT · Solar, Electricity, and Traffic | CRPS-sum: 0.301(.001) -> 0.253(.002); 0.024(.000) -> 0.021(.001); 0.050(.003) -> 0.040(.000) | Helped |
| add classifier-free guidance (ablation) | LDT · Solar, Electricity, and Traffic | CRPS-sum: 0.264(.004) -> 0.253(.002); 0.023(.003) -> 0.021(.001); 0.047(.004) -> 0.040(.000) | Helped |
Surprising findings
- Predicting the noise epsilon performs much worse than predicting the clean latent x0 in the non-autoregressive setting, unlike autoregressive TimeGrad where epsilon works.
Limitations
- Guidance strength must be tuned per dataset, with larger w better for deterministic datasets and smaller w for Solar and Taxi.
- Some baselines fail or run out of memory (CSDI on Traffic/Taxi/Wikipedia), so comparisons are incomplete.
Experimental setup
Datasets: Solar Electricity Traffic Taxi Wikipedia
Benchmarks: Solar Electricity Traffic Taxi Wikipedia
Metrics: CRPS-sum MSE
Baselines: GP KVAE Vec-LSTM-ind-scaling GP-Copula LSTM-MAF HMGT TLAE TimeGrad CSDI SSSD D3VAE
Builds on: Latent Diffusion Models DDPM DDIM classifier-free guidance TimeGrad
Closest papers by meaning
- TimeDART: A Diffusion Autoregressive Transformer for Self-Supervised Time Series Representation (ICML 2025 · similarity 0.91)
- HDT: Hierarchical Discrete Transformer for Multivariate Time Series Forecasting (AAAI 2025 · similarity 0.90)
- Diffusion-TS: Interpretable Diffusion for General Time Series Generation (ICLR 2024 · similarity 0.89)
- Non-stationary Diffusion For Probabilistic Time Series Forecasting (ICML 2025 · similarity 0.89)
- Considering Nonstationary within Multivariate Time Series with Variational Hierarchical Transformer for Forecasting (AAAI 2024 · similarity 0.88)
- Conditional Diffusion Model with Nonlinear Data Transformation for Time Series Forecasting (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.