Research map / Time series / Long-term forecasting
CARD: Channel Aligned Robust Blend Transformer for Time Series Forecasting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CARD, an encoder-only Transformer that adds channel-aligned attention (over tokens and over hidden dimensions, with dynamic projection across channels), a token blend module for multi-scale tokens, and a signal-decay robust (Laplace-based) loss for long-term time series forecasting.
Problem
Channel-independent Transformers for multivariate time series forecasting ignore cross-channel correlations, while channel-dependent designs overfit noise and are costly; multi-scale information and overfitting to noise are also not handled well.
Approach
Attention is applied alternately over tokens and over channels (with dynamic projection to reduce channel cost) and over hidden dimensions within each patch; a token blend module merges adjacent tokens within the same head to create multi-resolution tokens; the loss weights far-horizon errors by l^{-1/2} using an L1 form derived from a Markov/Laplace argument.
Key results
- CARD achieves average MSE/MAE of 0.383/0.384 on ETTm1 and 0.168/0.258 on Electricity (lookback 96), best in 6 of 7 datasets for MSE and 7 of 7 for MAE.
- CARD reaches average F1 of 0.890 on five anomaly detection benchmarks, outperforming the best baseline by about 3%.
- The signal decay loss reduces MSE by 3% to 12% when plugged into baselines such as MICN, TimesNet, FEDformer and Autoformer.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use signal decay-based loss instead of MSE (ablation) | CARD · ETTm1 | MSE: 0.390 -> 0.383 | Helped |
| Use signal decay-based loss instead of MSE (ablation) | CARD · ETTh1 | MSE: 0.449 -> 0.443 | Helped |
| Use signal decay-based loss instead of MSE (ablation) | MICN-regre · ETTm1 | MSE: 0.392 -> 0.383 | Helped |
| Use signal decay-based loss instead of MSE (ablation) | TimesNet · ETTm1 | MSE: 0.400 -> 0.392 | Helped |
| Use signal decay-based loss instead of MSE (ablation) | FEDformer · ETTm1 | MSE: 0.448 -> 0.413 | Helped |
| Use signal decay-based loss instead of MSE (ablation) | Autoformer · ETTm1 | MSE: 0.588 -> 0.523 | Helped |
| Increase input length from 96 to 720 (ablation) | CARD · ETTm1 | MSE: 8.9% reduction | Helped |
| Increase input length from 96 to 720 (ablation) | CARD · ETTh1 | MSE: 8.6% reduction | Helped |
| Set token blend size to 1, reducing token blend to standard token mixing (ablation) | CARD · Forecasting benchmarks | MSE | Hurt |
| Increase token blend size beyond the dataset-useful scales (ablation) | CARD · Some forecasting datasets | MSE | Hurt |
| Add signal decay-based loss instead of plain MSE (ablation) | Recent Transformer, CNN, and MLP baselines · Long-term forecasting datasets | MSE: 3% to 12% reduction | Helped |
Surprising findings
- Plugging the signal decay loss into other baselines improves them, and the gains are largest for frequency-domain models such as FEDformer and Autoformer.
Limitations
- Channel attention cost grows with the number of covariates, which is mitigated only by dynamic projection.
- Larger token blend sizes can oversmooth and hurt performance on some datasets.
Experimental setup
Datasets: ETTm1 ETTm2 ETTh1 ETTh2 Weather Electricity Traffic
Benchmarks: ETTm1 ETTm2 ETTh1 ETTh2 Weather Electricity Traffic SMD MSL SMAP SWaT PSM
Metrics: MSE MAE F1
Baselines: PatchTST MICN TimesNet Crossformer Dlinear LightTS FilM ETSformer FEDformer
Builds on: PatchTST Crossformer RevIN FlashAttention
Closest papers by meaning
- CSformer: Combining Channel Independence and Mixing for Robust Multivariate Time Series Forecasting (AAAI 2025 · similarity 0.87)
- SAMformer: Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Minimization and Channel-Wise Attention (ICML 2024 · similarity 0.87)
- Revisiting the Generic Transformer: Deconstructing a Strong Baseline for Time Series Foundation Models (NeurIPS 2026 · similarity 0.87)
- A Closer Look at Transformers for Time Series Forecasting: Understanding Why They Work and Where They Struggle (ICML 2025 · similarity 0.86)
- iTransformer: Inverted Transformers Are Effective for Time Series Forecasting (ICLR 2024 · similarity 0.86)
- DeformableTST: Transformer for Time Series Forecasting without Over-reliance on Patching (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.