Research map / Time series / Long-term forecasting
CSformer: Combining Channel Independence and Mixing for Robust Multivariate Time Series Forecasting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CSformer, a Transformer for multivariate time series forecasting that applies channel-wise then sequence-wise multiheaded self-attention with shared parameters, plus channel and sequence adapters and a dimension-augmented embedding.
Problem
Channel-independent models discard inter-channel interactions and channel-mixing models introduce noise; neither is optimal for multivariate time series forecasting.
Approach
Channel independence followed by mixing: a two-stage MSA (channel MSA then sequence MSA) with shared weights, reshaped between stages, plus adapters after each stage and a dimension-augmented (per-point learnable vector) embedding that avoids patching.
Key results
- CSformer achieves best average MSE on ETTh1 (0.429 vs 0.454 for iTransformer), ETTh2 (0.364), Electricity (0.176) and Solar Energy (0.230) in the main table.
- Top-1 count in main table: 6 MSE and 4 MAE wins out of 14 entries across datasets, reported as state-of-the-art overall.
- Replacing channel-independent attention with channel mixing in PatchTST improves Electricity MSE by 17.1% (0.216 to 0.179).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CSformer instead of the strongest baseline | multivariate forecasting models · ETTh1 | MSE: 0.440 -> 0.429 | Helped |
| Use CSformer instead of the strongest baseline | multivariate forecasting models · Electricity | MSE: 0.178 -> 0.176 | Helped |
| Use CSformer instead of the strongest baseline | multivariate forecasting models · Solar Energy | MAE: 0.262 -> 0.270 | Hurt |
| Add Channel MSA and its adapter (ablation) | CSformer · ETTh1 | MSE: 0.431 -> 0.429 | Helped |
| Add Sequence MSA and its adapter (ablation) | CSformer · Weather | MSE: 0.259 -> 0.250 | Helped |
| Share parameters between Channel MSA and Sequence MSA (ablation) | CSformer · ETTh1 | MSE: 0.440 -> 0.429 | Helped |
| Share parameters between Channel MSA and Sequence MSA (ablation) | CSformer · Weather | MSE: 0.256 -> 0.250 | Helped |
| Apply Channel MSA before Sequence MSA (ablation) | CSformer · ETTh1 | MSE: 0.443 -> 0.429 | Helped |
| Apply Channel MSA before Sequence MSA (ablation) | CSformer · Electricity | MSE: 0.148 -> 0.146; 0.164 -> 0.172 | Mixed |
| Add both adapters (ablation) | CSformer · Electricity | MSE: 0.192 -> 0.176 | Helped |
| Add Channel Adapter (ablation) | CSformer · Electricity | MSE: 0.201 -> 0.176 | Helped |
| Add Sequence Adapter (ablation) | CSformer · Electricity | MSE: 0.192 -> 0.176 | Helped |
Surprising findings
- iTransformer had the poorest cross-dataset generalization (ETTh2 to ETTh1 MSE around 0.88-0.92) in the authors' transfer experiment.
- Sequence-MSA-only variant performs competitively, which the authors attribute to implicit channel independence.
Limitations
- Authors note they have not yet investigated computational demand reduction.
- Performance gains are modest on several datasets (e.g., Weather where PatchTST and iTransformer are close), and it does not win on all metrics.
Experimental setup
Datasets: ETTh1 ETTh2 ETTm1 ETTm2 Weather Electricity Solar Energy
Benchmarks: ETTh1 ETTh2 ETTm1 ETTm2 Weather Electricity Solar Energy
Metrics: MSE MAE
Baselines: iTransformer PatchTST Crossformer SCINet TiDE TimesNet DLinear FEDformer Stationary Autoformer Informer
Builds on: Transformer iTransformer PatchTST Crossformer RevIN
Closest papers by meaning
- From Similarity to Superiority: Channel Clustering for Time Series Forecasting (NeurIPS 2024 · similarity 0.90)
- TimeCHEAT: A Channel Harmony Strategy for Irregularly Sampled Multivariate Time Series Analysis (AAAI 2025 · similarity 0.89)
- SOFTS: Efficient Multivariate Time Series Forecasting with Series-Core Fusion (NeurIPS 2024 · similarity 0.88)
- CARD: Channel Aligned Robust Blend Transformer for Time Series Forecasting (ICLR 2024 · similarity 0.87)
- xLSTM-Mixer: Multivariate Time Series Forecasting by Mixing via Scalar Memories (NeurIPS 2025 · similarity 0.87)
- HyperMixer: Specializable Hypergraph Channel Mixing for Long-term Multivariate Time Series Forecasting (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.