Research map / Time series / Causal and generative time series
Glocal Information Bottleneck for Time Series Imputation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Glocal-IB, a model-agnostic information-bottleneck training paradigm that adds a KL regularizer and a global alignment loss (one MLP) to reconstruction training to improve time series imputation under high missing rates.
Problem
Time series imputation models trained with point-wise reconstruction losses overfit local noise under high missing rates, yielding low training loss but poor inference imputation and distorted latent representations.
Approach
Extends the standard information bottleneck objective with a KL-based compactness term on the encoder posterior and a contrastive-derived Global Alignment loss that aligns latent representations of masked inputs with those of their fully observed counterparts, using only one extra MLP projector.
Key results
- Glocal-IB on a vanilla Transformer achieves the lowest MAE and MSE on all nine datasets averaged over missing rates, e.g., ETTh1 MSE 0.197 vs. 0.376 for SAITS (38% missing average).
- On ETTh1 at 90% missing rate, Glocal-IB Transformer reaches MAE 0.506 / MSE 0.497 versus 0.585 / 0.670 for DLinear (second best).
- Glocal-IB adds only one MLP and shows marginal memory/time overhead compared to original models, while Time-MoE alignment is substantially more costly; Time-MoE alignment gives only marginal gains.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Glocal-IB training, including global alignment and regularization, to a vanilla Transformer | Transformer · Nine time-series datasets | MAE | Helped |
| Add Glocal-IB training, including global alignment and regularization, to a vanilla Transformer | Transformer · Nine time-series datasets | MSE | Helped |
| Add the regularization loss (ablation) | Glocal-IB · ETTh1, ETTh2, ETTm1, and ETTm2 | MAE | Helped |
| Add the Global Alignment loss (ablation) | Glocal-IB · ETTh1, ETTh2, ETTm1, and ETTm2 | MAE | Helped |
| Add the regularization loss instead of using only local reconstruction (ablation) | Glocal-IB · ETT datasets | MAE | Helped |
| Increase the regularization weight beyond 0.01 (ablation) | Glocal-IB · ETT datasets | MAE | Hurt |
| Align representations with Time-MoE rather than use Glocal-IB | TimesNet and SAITS · ETT datasets | MAE | No clear effect |
Surprising findings
- Low training loss under high missing rates does not imply good inference-time imputation; fewer training epochs with higher training loss can yield better imputation.
- Time-series foundation model (Time-MoE) representation alignment provides only limited benefit for imputation.
Limitations
- Evaluated only on three backbones (TimesNet, SAITS, Transformer) and four ETT datasets in the generality analysis, due to computational constraints.
- Performance gain at extreme 90% missingness is less pronounced because the preserved global structure is weak with only 10% observed data.
- Gaussian likelihood with fixed variance and several approximations (e.g., InfoNCE simplification to an alignment loss) make the MI derivations approximate.
Experimental setup
Datasets: ETTh1 ETTh2 ETTm1 ETTm2 Beijing Air PEMS-Traffic Electricity Weather Metr-LA
Benchmarks: ETTh1 ETTh2 ETTm1 ETTm2 Beijing Air PEMS-Traffic Electricity Weather Metr-LA
Metrics: MAE MSE
Baselines: SAITS Transformer DLinear TimesNet FreTS PatchTST iTransformer GPVAE TimeMixer
Builds on: information bottleneck InfoNCE SAITS TimesNet PyPOTS
Closest papers by meaning
- ProCTI: Prototype-Refined Global Conditioning for Diffusion-Based Time Series Imputation (NeurIPS 2026 · similarity 0.88)
- BayOTIDE: Bayesian Online Multivariate Time Series Imputation with Functional Decomposition (ICML 2024 · similarity 0.87)
- Discretizing Continuous Time Series for Imputation with Masked Diffusion Training (NeurIPS 2026 · similarity 0.86)
- Learning from Highly Sparse Spatio-temporal Data (NeurIPS 2024 · similarity 0.86)
- Frequency-aware Generative Models for Multivariate Time Series Imputation (NeurIPS 2024 · similarity 0.85)
- Diffusion Transformers for Imputation: Statistical Efficiency and Uncertainty Quantification (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.