Research map / Time series / Causal and generative time series
A Diffusion Model for Regular Time Series Generation from Irregular Data with Completion and Masking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a two-step framework that completes irregular time series with a Time Series Transformer autoencoder and then trains an ImagenTime-style vision diffusion model on delay-embedding images with a loss masked to observed/completed-pixel regions, yielding state-of-the-art generation on irregular benchmarks.
Problem
Generating regular (regularly sampled) time series from irregularly sampled, partially missing data; prior methods (GT-GAN, KoVAE) rely on costly NCDE preprocessing and GAN/VAE backbones, and naive zero-filled masking of diffusion models creates unnatural pixel neighborhoods.
Approach
Complete irregular series with a TST autoencoder to create natural image neighborhoods, then train an ImagenTime diffusion model with a loss masked to the originally observed pixels so completed values act only as weak conditioning; also modifies the delay-embedding inverse to average over all corresponding pixels.
Key results
- Averaged over missing rates at length 24, the method improves discriminative score by 74.2% relative to the second-best method (typically KoVAE), with context-FID improvement 78.5%.
- Training time at length 24 is about 6.5x faster than KoVAE on average, with discriminative score about 3.4x better.
- On ultra-long KDD-Cup sequences (10,920), discriminative score improves 36.6% over KoVAE (0.155 vs 0.375 at 30% missing).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TST-based completion followed by masked diffusion rather than KoVAE | regular time-series generation from irregular data · ETTh1, length 24 | discriminative score: 0.197 -> 0.037 | Helped |
| Use TST-based completion followed by masked diffusion rather than KoVAE | regular time-series generation from irregular data · ETTh2, length 24 | discriminative score: 0.081 -> 0.009 | Helped |
| Use TST-based completion followed by masked diffusion rather than KoVAE | regular time-series generation from irregular data · Electricity, length 24 | discriminative score: 0.498 -> 0.384 | Helped |
| Use TST-based completion followed by masked diffusion rather than KoVAE | regular time-series generation from irregular data · KDD-Cup, length 10,920, 30% missing | discriminative score: 0.375 -> 0.155 | Helped |
| Add TST-based completion before masked diffusion (ablation) | ImagenTime · Energy, length 24, 30% missing | discriminative score: 0.157 -> 0.048 | Helped |
| Add masking during diffusion training (ablation) | TST-completed ImagenTime · Energy, length 24, 50% missing | discriminative score: 0.307 -> 0.065 | Helped |
| Add masking during diffusion training (ablation) | TST-completed ImagenTime · Stock, length 24, 30% missing | discriminative score: 0.025 -> 0.007 | Helped |
| Replace KoVAE's NCDE imputation with TST imputation (ablation) | KoVAE · Energy and Stock, lengths 24, 96, and 768 | discriminative score | Mixed |
| Use TST completion with TimeAutoDiff instead of the proposed diffusion setup (ablation) | TimeAutoDiff · Energy and Stock, lengths 24, 96, and 768 | discriminative score | Hurt |
| Use TST completion with TransFusion instead of the proposed diffusion setup (ablation) | TransFusion · Energy and Stock, lengths 24, 96, and 768 | discriminative score | Hurt |
| Average all image pixels corresponding to each time point rather than use only the first pixel (ablation) | inverse delay embedding · ETTh1, length 24, 30% missing | discriminative score: 0.023 -> 0.020 | Helped |
| Average all image pixels corresponding to each time point rather than use only the first pixel (ablation) | inverse delay embedding · ETTh2, length 24, 50% missing | discriminative score: 0.037 -> 0.005 | Helped |
Surprising findings
- Plain masking of zero-filled missing values barely improves over unmasked training in the toy and Stocks experiments (score-estimation loss 0.71 vs 0.67 toy; 0.81 to 0.79 Stocks).
- Replacing KoVAE's NCDE with TST in KoVAE does not close the performance gap, suggesting the completion-plus-masking design, not the imputer alone, drives gains.
Limitations
- Relies on an autoencoder completion step whose errors may still propagate, although masking limits reliance on completed values.
- Predictive gains are smaller than discriminative gains (about 15% average improvement), and some settings (e.g., Electricity at length 24) are not best.
Experimental setup
Datasets: Sines Stocks MuJoCo Energy ETTh1 ETTh2 ETTm1 ETTm2 Weather Electricity KDD-Cup Traffic
Benchmarks: Sines Stocks MuJoCo Energy ETTh1 ETTh2 ETTm1 ETTm2 Weather Electricity KDD-Cup
Metrics: discriminative score predictive score (MAE) Context-FID correlation score training time inference time
Baselines: KoVAE GT-GAN TimeGAN-Δt TimeAutoDiff TransFusion CSDI
Builds on: ImagenTime KoVAE GT-GAN EDM TST
Closest papers by meaning
- Utilizing Image Transforms and Diffusion Models for Generative Modeling of Short and Long Time Series (NeurIPS 2024 · similarity 0.92)
- Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach (NeurIPS 2025 · similarity 0.89)
- Discretizing Continuous Time Series for Imputation with Masked Diffusion Training (NeurIPS 2026 · similarity 0.88)
- TimeDART: A Diffusion Autoregressive Transformer for Self-Supervised Time Series Representation (ICML 2025 · similarity 0.87)
- Diffusion-TS: Interpretable Diffusion for General Time Series Generation (ICLR 2024 · similarity 0.86)
- Learning to Embed Time Series Patches Independently (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.