Research map / Time series / Long-term forecasting

A Set-Sequence Model for Time Series

NeurIPS 2026 · Time series › Long-term forecasting · approach: Graph-based multivariate forecasting · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Set-Sequence, which adds a permutation-invariant set module at each time step to summarize the cross-section of units and feeds the augmented per-unit features into a standard sequence backbone, evaluated on a synthetic contagion task, equity portfolio optimization, and mortgage risk prediction.

Problem

Predicting outcomes for large cross-sections of exchangeable time series (loans, stocks, sensors) where cross-unit dependencies are usually captured only via hand-crafted summary features, and joint modeling of all units is intractable.

Approach

A per-time-step mean-pooled set summary (embedding φ, mean aggregation, projection ρ) is concatenated to each unit's features and passed to a per-unit sequence layer with shared weights, learning cross-sectional structure with linear cost in the number of units and supporting variable unit counts at inference.

Key results

• On the synthetic task, Set-Sequence with LongConv reduces KL from 0.0018 (single-unit) to 0.00018 (10.2x) and raises AUC from 0.757 to 0.802.
• On equities, Set-Sequence achieves an annualized Sharpe ratio of 4.82 ± 0.12 versus 3.94 for the best sequence baseline (S4), and outperforms the CNN-Transformer by 42% on the 2002-2016 aligned period.
• On mortgage risk, Set-Sequence improves average AUC by about 4 points over the best domain-specific baseline (5-layer neural network) and wins on 22 of 25 transitions.

Surprising findings

• Joint (concatenated full cross-section) sequence models underperform even single-unit modeling on the synthetic task, showing that naive joint modeling fails at large cross-sections.
• On the synthetic task, the simple linear set aggregation achieves accuracy comparable to multi-head attention at lower cost, with MHA improving AUC only slightly (0.815 vs 0.802) while costing 3.3x time and 3.5x memory.

Limitations

• Mortgage data restricted to four ZIP codes in the greater LA area, limiting generality.
• Equity results depend on a CRSP-based universe and assume a fixed set of 500 largest-cap stocks per period.
• Set summary uses mean aggregation, and the paper notes MHA variant may be better when cross-unit dependencies are strong.

Experimental setup

Datasets: synthetic contagion data (simulated) CRSP equity returns CoreLogic Loan-Level Market Analytics

Benchmarks: synthetic contagion task equity portfolio construction (Jan 2002-Dec 2021) mortgage state transition prediction (2010-2022 test)

Metrics: KL divergence AUC annualized Sharpe ratio cross-entropy loss average AUC correlation with latent factor R^2

Baselines: Transformer S4 H3 Hyena LongConv CNN-Transformer (IPCA) 5-layer neural network (mortgage) logistic regression (mortgage) Kalman Filter

Builds on: Deep Sets Set Transformer LongConv S4 H3

Closest papers by meaning

• Market-GAN: Adding Control to Financial Market Data Generation with Semantic Context (AAAI 2024 · similarity 0.85)
• Chimera: Effectively Modeling Multivariate Time Series with 2-Dimensional State Space Models (NeurIPS 2024 · similarity 0.85)
• STEM-LTS: Integrating Semantic-Temporal Dynamics in LLM-driven Time Series Analysis (AAAI 2025 · similarity 0.85)
• A Closer Look at Transformers for Time Series Forecasting: Understanding Why They Work and Where They Struggle (ICML 2025 · similarity 0.85)
• Towards Financial World Modeling (NeurIPS 2026 · similarity 0.85)
• An Analysis of Linear Time Series Forecasting Models (ICML 2024 · similarity 0.84)