Research map / Time series / Causal and generative time series
Diffusion Transformers for Imputation: Statistical Efficiency and Uncertainty Quantification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes conditional diffusion transformers for time-series imputation on Gaussian process data, proving sample-complexity and confidence-region coverage bounds that depend on missing-pattern condition numbers, and proposes a mixed-masking training strategy.
Problem
Theoretical understanding of how well diffusion transformers learn the conditional distribution of missing values given observed values, how missing patterns affect sample efficiency, and whether constructed confidence regions achieve nominal coverage.
Approach
Represents the closed-form conditional score of a Gaussian process as the output of nested gradient descent (algorithm unrolling), shows transformers can approximate each unrolled step, and derives TV-distance sample complexity and coverage bounds scaling with condition numbers of the conditional covariance; motivated by this, trains on a mixture of masking patterns (mixed-masking).
Key results
- On the latent GP data with mixed-masking S4, DiT reaches MSE 0.53 on pattern P4 versus 0.59 for CSDI under random masking-level comparison, with DiT outperforming CSDI and GPVAE in MSE across patterns.
- Clustered missingness gives condition number 415.40 versus 3.00 for dispersed missingness on the stated GP example, indicating missing pattern strongly affects difficulty.
- Mixed-masking S4 reduces the empirical distribution-shift ratio by about 47.93x relative to purely random masking (S1) for strongly grouped test patterns.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace S1 random-mask training with mixed-mask training (ablation) | DiT · Latent GP P1 | MSE: 0.70, 0.03 -> 0.67, 0.03 | Helped |
| Replace S1 random-mask training with mixed-mask training (ablation) | DiT · Latent GP P2 | MSE: 0.64, 0.03 -> 0.60, 0.03 | Helped |
| Replace S1 random-mask training with mixed-mask training (ablation) | DiT · Latent GP P3 | MSE: 0.62, 0.02 -> 0.58, 0.02 | Helped |
| Replace S1 random-mask training with mixed-mask training (ablation) | DiT · Latent GP P4 | MSE: 0.56, 0.01 -> 0.53, 0.01 | Helped |
| Replace S1 random-mask training with mixed-mask training (ablation) | DiT · Latent GP P1 | CR coverage (%): 36.46, 1.62 -> 56.43, 3.76 | Helped |
| Replace S1 random-mask training with mixed-mask training (ablation) | DiT · Latent GP P2 | CR coverage (%): 55.81, 1.55 -> 66.24, 3.22 | Helped |
| Replace S1 random-mask training with mixed-mask training (ablation) | DiT · Latent GP P3 | CR coverage (%): 63.53, 1.72 -> 73.36, 4.37 | Helped |
| Replace S1 random-mask training with mixed-mask training (ablation) | DiT · Latent GP P4 | CR coverage (%): 76.46, 1.33 -> 80.64, 3.72 | Helped |
| Use DiT rather than CSDI | Latent GP imputation · Latent GP P1 | MSE: 0.75, 0.03 -> 0.70, 0.03 | Helped |
| Use DiT rather than CSDI | Latent GP imputation · Latent GP P2 | MSE: 0.66, 0.03 -> 0.64, 0.03 | Helped |
| Use DiT rather than CSDI | Latent GP imputation · Latent GP P3 | MSE: 0.65, 0.02 -> 0.62, 0.02 | Helped |
| Use DiT rather than CSDI | Latent GP imputation · Latent GP P4 | CR coverage (%): 68.60, 1.74 -> 76.46, 1.33 | Helped |
Surprising findings
- CR coverage for DiT under purely random masking (S1) on pattern P1 was only about 36%, far below the 95% nominal level, improving to about 56% with mixed masking.
Limitations
- Theory restricted to Gaussian process data and MCAR block-missing setting.
- Optimal mixing ratios for mixed-masking are instance-dependent and left open.
- Experiments are only on synthetic GP data; real-world datasets are deferred to appendix.
Experimental setup
Datasets: synthetic Gaussian process data (H=96, d=8) latent Gaussian process data (nonlinear transform of GP plus noise)
Benchmarks: synthetic Gaussian process data latent Gaussian process data
Metrics: TV distance (theory) MSE CR coverage rate
Baselines: CSDI GPVAE
Builds on: CSDI DiT score-based diffusion models
Closest papers by meaning
- Discretizing Continuous Time Series for Imputation with Masked Diffusion Training (NeurIPS 2026 · similarity 0.92)
- ProCTI: Prototype-Refined Global Conditioning for Diffusion-Based Time Series Imputation (NeurIPS 2026 · similarity 0.91)
- Rethinking the Diffusion Models for Missing Data Imputation: A Gradient Flow Perspective (NeurIPS 2024 · similarity 0.90)
- DiffPuter: Empowering Diffusion Models for Missing Data Imputation (ICLR 2025 · similarity 0.90)
- RDPI: A Refine Diffusion Probability Generation Method for Spatiotemporal Data Imputation (AAAI 2025 · similarity 0.90)
- Frequency-aware Generative Models for Multivariate Time Series Imputation (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.