Research map / Probabilistic and scientific modeling / Variational and generative sampling
Sequence Modeling with Spectral Mean Flows
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes spectral mean flows: an operator-theoretic HMM embedding of sequence distributions in tensor product RKHS, trained via flow matching on a tensor-network (EVD-based) parameterization, for time-series generation.
Problem
Neural sequence modeling of nonlinear probabilistic state dynamics; naive MMD gradient flows over sequences have exponentially large tensor embeddings and slow sampling convergence.
Approach
Embed the full sequence distribution as a tensor in a product RKHS, decompose it via spectral (eigen) decomposition of the HMM transition CME operator into a tensor network with linear cost in sequence length, and extend MMD gradient flows to time-dependent RKHS connected to flow matching so sampling converges at t=1.
Key results
- Best context-FID, correlational and discriminative scores in 18 of 24 regular time-series cases versus Diffusion-TS and other baselines.
- Peak GPU memory experiment: without tensor network decomposition the inner product is almost infeasible for N>4 (d=32).
- Best discriminative score on irregular Stocks settings (e.g., 0.009 at 0% drop vs 0.021 for Koopman VAE).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use spectral mean flow instead of Diffusion-TS | Time-series generative modeling · Sines, Stocks, ETTh, MuJoCo, Energy, and fMRI | Context-FID Score: 0.013±.001 -> 0.004±.001; 0.169±.021 -> 0.008±.003; 0.126±.007 -> 0.058±.007; 0.015±.001 -> 0.018±.002; 0.113±.011 -> 0. | Helped |
| Train spectral mean flow with the flow-matching approach | Spectral Mean Flow · 2D checkerboard | MMD to target over sampling time | Helped |
| Add a stability loss (ablation) | Spectral Mean Flow · Nonlinear pendulum | Correlation Score: 0.0029±.0008 -> 0.0005±.0004 | Helped |
Surprising findings
- Model is trained only with the standard flow matching loss, without auxiliary time-series-specific losses such as Fourier-domain losses.
Limitations
- Assumes well-specified CME operators and RKHS closure under pointwise multiplication (Sobolev spaces with s > d/2), which may not hold in practice.
- Complex-valued spectral expansion is not fully covered by the existing theory; authors note it is out of scope.
- Sampling mainly validated on short time series (length 24) and limited larger-scale experiments.
Experimental setup
Datasets: Sines Stocks ETTh MuJoCo Energy fMRI FRED-MD NN5 Daily 2D checkerboard
Benchmarks: Sines Stocks ETTh MuJoCo Energy fMRI FRED-MD NN5 Daily
Metrics: context-FID correlational score discriminative score predictive score marginal score classification score
Baselines: Diffusion-TS DiffTime Diffwave TimeGAN TimeVAE Cot-GAN RNN-AR SDFormer-AR SDFormer-M ImagenTime LS4 SaShiMi-AR
Builds on: flow matching MMD gradient flow conditional mean embeddings Koopman operator theory Diffusion-TS
Closest papers by meaning
- MaRK: Markov-adapted Recurrent Kernels for Dynamic Operator Conditioning in State Space Models (NeurIPS 2026 · similarity 0.86)
- Inferring stochastic low-rank recurrent neural networks from neural data (NeurIPS 2024 · similarity 0.85)
- Sobolev Regularized MMD Gradient Flow (NeurIPS 2026 · similarity 0.85)
- Asymptotics of SGD in Sequence-Single Index Models and Single-Layer Attention Networks (NeurIPS 2025 · similarity 0.85)
- Structured Linear CDEs: Maximally Expressive and Parallel-in-Time Sequence Models (NeurIPS 2025 · similarity 0.85)
- Efficient Training of Neural Stochastic Differential Equations by Matching Finite Dimensional Distributions (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.