Research map / Time series / Long-term forecasting

Entropy Guided Dynamic Patch Segmentation for Time Series Transformers

NeurIPS 2026 · Time series › Long-term forecasting · approach: Attention for time series forecasting

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes EntroPE, which places patch boundaries in time series at peaks of conditional entropy from a frozen tiny causal transformer over quantized values, and feeds variable-length patches into a PatchTST-style transformer with cross-attention patch encoding.

Problem

Fixed-length, temporally agnostic patching in time series transformers splits natural transitions across patch boundaries and causes train-inference mismatch in patch configurations.

Approach

Use per-timestep conditional entropy from a frozen ~10k-parameter causal transformer trained on quantized series to place patch boundaries at high-uncertainty points (sample-adaptive quantile thresholds), then encode variable-length patches with max pooling plus within-patch cross-attention and a fusion decoder.

Key results

• Forecasting: EntroPE reports best MSE on ETTh1 (0.416 vs 0.427 for TimeMosaic) and ETTh2 (0.366), and on Weather (0.242), averaged over horizons 96-720 with input length 96.
• Classification: average accuracy 77.13% over 10 UEA datasets, versus 76.03% for TSLANet and 71.14% for PatchTST.
• Anomaly detection: average F1 87.72% versus 87.54% for TSLANet; EntroPE is not best on SMD, SMAP, or PSM.

Surprising findings

• On Electricity, CALF and iTransformer outperform EntroPE (0.182 MSE for EntroPE vs 0.177 for CALF), so the method does not win everywhere.
• A single-token (patch length 1) static configuration remains competitive with dynamic patching in the ablation.

Limitations

• Entropy model is quantized and channel-independent; quantization range and vocabulary size choices affect boundaries.
• Gains are modest and not uniform; EntroPE underperforms baselines on Electricity and several anomaly datasets.
• Claims of 500-1000x fewer parameters compare against different baseline scales and reported efficiency is only shown on ETTm1.

Experimental setup

Datasets: ETTh1 ETTh2 ETTm1 ETTm2 Weather Electricity 10 UEA multivariate classification datasets SMD MSL SMAP SWaT PSM

Benchmarks: ETTh1 ETTh2 ETTm1 ETTm2 Weather Electricity UEA 10 datasets SMD MSL SMAP SWaT PSM

Metrics: MSE MAE accuracy F1

Baselines: TimeMosaic iTransformer PatchTST FEDformer Autoformer TimesNet TSLANet TimeMixer HDMixer DLinear Time-FFM TimeBase

Builds on: PatchTST Byte Latent Transformer Perceiver Chronos GPT-2

Closest papers by meaning

• DeformableTST: Transformer for Time Series Forecasting without Over-reliance on Patching (NeurIPS 2024 · similarity 0.87)
• TimeDART: A Diffusion Autoregressive Transformer for Self-Supervised Time Series Representation (ICML 2025 · similarity 0.87)
• Multi-Patch Prediction: Adapting Language Models for Time Series Representation Learning (ICML 2024 · similarity 0.86)
• Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shift (NeurIPS 2025 · similarity 0.86)
• Unlocking the Power of Patch: Patch-Based MLP for Long-Term Time Series Forecasting (AAAI 2025 · similarity 0.86)
• Learning to Embed Time Series Patches Independently (ICLR 2024 · similarity 0.86)