Research map / Time series / Long-term forecasting

Spectral Transformer Neural Processes

NeurIPS 2026 · Time series › Long-term forecasting · approach: Spatiotemporal forecasting models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Spectral Transformer Neural Processes (STNPs), which add a context-inferred spectral-mixture embedding to TNPs so attention can exploit periodic and quasi-periodic structure.

Problem

Neural processes underfit and generalise poorly on data with strong periodicity or quasi-periodicity, because their time-domain embeddings do not encode frequency structure.

Approach

A Spectral Aggregator estimates an empirical spectrum from the context set, compresses it into a learned Gaussian spectral mixture via a CNN responsibility network, samples task-adaptive frequencies to form Fourier features, and concatenates them with MLP token embeddings. In expectation this injects a spectral-mixture kernel bias into TNP attention without changing the attention mechanism.

Key results

• On DTD image completion, STNP reaches PSNR 24.0 and SSIM 0.64 versus 23.6 and 0.62 for SConvCNP.
• On the periodic-kernel GP benchmark with m_min=20, the full STNP attains target log-likelihood 1.14, while the FAN-style sinusoidal embedding variant reaches -0.35.
• On ETTh1, STNP achieves MAE 0.487 and MSE 0.539, versus 0.512 and 0.592 for the sinusoidal-embedding ablation.

Surprising findings

• Adding sinusoidal features alone (FAN-style embedding) performs poorly on periodic regression, so the gains do not come only from sinusoidal features.
• TETNP could not be trained on DTD image completion due to out-of-memory failures even with 8 A100 GPUs.

Limitations

• Selecting the frequency grid size and component-sample allocation is not task-adaptive and is left open.
• Chichester Harbour is trained with an L2 objective, so its predictive bands are only qualitative uncertainty estimates.
• TETNP could not be run on image completion due to memory limits, so comparisons there are incomplete.

Experimental setup

Datasets: synthetic GP functions (RBF, Matern, periodic kernels) sawtooth functions DTD Electricity ETTh1 Exchange Rate National Illness Traffic Weather California Traffic Flow Chimet Chichester Harbour

Benchmarks: periodic-kernel GP DTD image completion six-dataset forecasting benchmark California Traffic Flow Chichester Harbour ETTh1

Metrics: log-likelihood RMSE PSNR SSIM MAE MSE

Baselines: CNP NP BNP ANP CANP BANP TNP TETNP ConvCNP SConvCNP

Builds on: Transformer Neural Processes Fourier Analysis Networks random Fourier features spectral mixture kernel SConvCNP

Closest papers by meaning

• Spectral Convolutional Conditional Neural Process (NeurIPS 2025 · similarity 0.87)
• Gridded Transformer Neural Processes for Spatio-Temporal Data (ICML 2025 · similarity 0.87)
• Translation Equivariant Transformer Neural Processes (ICML 2024 · similarity 0.85)
• SimPSI: A Simple Strategy to Preserve Spectral Information in Time Series Data Augmentation (AAAI 2024 · similarity 0.85)
• TSLANet: Rethinking Transformers for Time Series Representation Learning (ICML 2024 · similarity 0.85)
• Rough Transformers: Lightweight and Continuous Time Series Modelling through Signature Patching (NeurIPS 2024 · similarity 0.85)