Research map / Time series / Long-term forecasting
A Lightweight Sparse Interaction Network for Time Series Forecasting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LSINet, an MLP-only linear forecaster that learns shared, sparsity-induced Bernoulli connection matrices between time patches (MSIM + SIL) to capture explicit temporal interactions cheaply.
Problem
Linear models for long-term time series forecasting capture temporal dependencies only implicitly through stacked MLPs, while transformers with self-attention are expensive; the goal is a lightweight linear model with explicit, efficient temporal interaction.
Approach
Replace self-attention score computation with a learned 0-1 connection matrix sampled from a Bernoulli distribution (Gumbel-Softmax relaxed), sparsified by a self-adaptive regularization loss that supervises the top-K connections, and shared across samples and variables (SIL) to reduce cost.
Key results
- LSINet achieves best MSE on most ETT and Weather/Electricity settings, e.g., ETTh1-96 MSE 0.366 vs PatchTST 0.37 and CI-TSMixer 0.373.
- Reports average MSE improvements of 2.06% over CI-TSMixer and 2.22% over PatchTST (avg across datasets and horizons).
- Efficiency: on Electricity (input 1024), LSINet trains at 91.25 s/epoch and uses 6936 MB vs PatchTST 377.25 s/epoch and 16128 MB (input 512).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add MSIM to the linear forecasting model (ablation) | LSINet · Weather and Electricity | MSE: 2.23% average and 6.54% maximum increase when removed | Helped |
| replace MSIM with self-attention (SAM) (ablation) | LSINet · Weather and Electricity | MSE: 2.24% average and 6.53% maximum increase | Hurt |
| replace MSIM with self-attention (SAM) (ablation) | LSINet · Weather and Electricity | training time, inference time, and memory usage: Weather: 22.36 -> 56.45 training time; 20.50 -> 50.59 inference time; 3026 -> 7584 memory | Hurt |
| add ASRL to the interaction learning component (ablation) | LSINet · Weather and Electricity | MSE: 1%, 1.86%, 2.25% and 1% | Helped |
| use LSINet instead of PatchTST | Weather · Weather | training time (s/epoch): 59.98 -> 18.92 | Helped |
| use LSINet instead of PatchTST | Weather · Weather | memory (MB): 5034 -> 2940 | Helped |
Surprising findings
- Replacing MSIM with standard self-attention in the linear model increases training time and memory substantially (e.g., Electricity 291.47 s/epoch and 23464 MB vs 91.25 s and 6936 MB) without accuracy gain.
Limitations
- Full hyperparameter and training details deferred to a full version not included here.
- Gains over strong baselines are small (about 2%), and some settings (e.g., Weather-96 MAE, ETTh2-720 MAE) are not best.
- Interaction matrix is static at inference and depends on a fixed max patch count (N~64), which may limit generality.
Experimental setup
Datasets: ETTh1 ETTh2 ETTm1 ETTm2 Weather Electricity
Benchmarks: ETTh1 ETTh2 ETTm1 ETTm2 Weather Electricity
Metrics: MSE MAE training time per epoch inference time GPU memory
Baselines: TimeMixer CI-TSMixer FiLM DLinear PatchTST Pathformer Scaleformer
Builds on: PatchTST Gumbel-Softmax TSMixer Informer
Closest papers by meaning
- TStruct: Learning Shared Temporal Structures for Long-Term Time-Series Forecasting (NeurIPS 2026 · similarity 0.89)
- Unlocking the Power of Patch: Patch-Based MLP for Long-Term Time Series Forecasting (AAAI 2025 · similarity 0.87)
- CFPT: Empowering Time Series Forecasting through Cross-Frequency Interaction and Periodic-Aware Timestamp Modeling (ICML 2025 · similarity 0.87)
- Affirm: Interactive Mamba with Adaptive Fourier Filters for Long-term Time Series Forecasting (AAAI 2025 · similarity 0.86)
- SparseTSF: Modeling Long-term Time Series Forecasting with *1k* Parameters (ICML 2024 · similarity 0.86)
- Sequence Complementor: Complementing Transformers for Time Series Forecasting with Learnable Sequences (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.