Research map / Architectures and efficiency / RNNs and normalization
Nonstationary Sparse Spectral Permanental Process
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NSSPP and deep DNSSPP, permanental (square-link Gaussian Cox) processes using sparse spectral features of nonstationary kernels with Laplace approximation, achieving linear-in-N complexity in the number of data points.
Problem
Permanental processes (square-link Gaussian Cox processes) have cubic cost in data size and are restricted to stationary or analytically tractable kernels, limiting expressiveness for nonstationary point process data.
Approach
Extend sparse spectral kernel approximation to nonstationary kernels via a two-frequency spectral representation with symmetrized, positive semi-definite spectral density, and stack spectral feature mappings into a deep kernel, with Laplace approximation for inference.
Key results
- On nonstationary synthetic data, DNSSPP-[100,50] achieves test log-likelihood 175.70 and RMSE 0.076, versus 165.94 for SSPP and 0.140 for SSPP RMSE.
- On Porto Taxi, DNSSPP-[30,50,30] reaches test log-likelihood 7246 (best), while LBPP reaches 6096.
- On Coal, DNSSPP-[50,30] gives the best test log-likelihood 225.85 versus 221.42 for SSPP; on Redwoods LBPP is best at 80.40 while DNSSPP is 79.5-79.6.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DNSSPP instead of the strongest stationary baseline DSSPP | Permanental processes on stationary synthetic data · Stationary Synthetic Data | test log-likelihood: 155.62($\pm $ 37.31) -> 156.55($\pm $ 37.44) | Helped |
| Use DNSSPP instead of DSSPP | Permanental processes on stationary synthetic data · Stationary Synthetic Data | RMSE: 0.060($\pm $ 0.015) -> 0.061($\pm $ 0.020) | No clear effect |
| Use DNSSPP instead of the strongest stationary baseline DSSPP | Permanental processes on nonstationary synthetic data · Nonstationary Synthetic Data | test log-likelihood: 174.35($\pm $ 26.70) -> 175.70($\pm $ 26.89) | Helped |
| Use DNSSPP instead of NSMPP | Permanental processes on nonstationary synthetic data · Nonstationary Synthetic Data | RMSE: 0.079($\pm $ 0.019) -> 0.076($\pm $ 0.015) | Helped |
| Use DNSSPP instead of NSMPP | Permanental processes on Coal Mining Disaster data · Coal Mining Disaster | test log-likelihood: 223.28($\pm$ 3.60) -> 225.85($\pm$ 2.61) | Helped |
| Use DNSSPP instead of LBPP | Permanental processes on Redwoods data · Redwoods | test log-likelihood: 80.40($\pm$ 0.72) -> 79.56($\pm$ 0.014) | Hurt |
| Use DNSSPP instead of NSMPP | Permanental processes on Porto Taxi data · Porto Taxi | test log-likelihood: 6492($\pm$ 62) -> 7246($\pm$ 37) | Helped |
| Increase network depth and width (ablation) | DNSSPP · Nonstationary Synthetic Data | test performance | Mixed |
| Use a single layer instead of a deeper DNSSPP network (ablation) | DNSSPP · Nonstationary Synthetic Data | test performance | Hurt |
| Increase network depth to four layers (ablation) | DNSSPP · Nonstationary Synthetic Data | test performance | Hurt |
| Increase network width on the small synthetic dataset (ablation) | DNSSPP · Nonstationary Synthetic Data | test performance | Hurt |
| Use a learning rate that is too low or too few optimization epochs (ablation) | DNSSPP hyperparameter optimization · Nonstationary Synthetic Data | test performance | Hurt |
Surprising findings
- On the stationary synthetic data, the shallow DSSPP and several stationary baselines perform comparably to DNSSPP, so the advantage is mainly under nonstationarity.
- DNSSPP has longer runtime than shallow models and cannot use analytic intensity integrals, relying on numerical integration.
Limitations
- Deep architecture prevents analytical intensity integral, requiring numerical integration that slows computation.
- Performance is sensitive to hyperparameters such as learning rate, epochs, width and depth (overfitting/underfitting).
- Experiments are limited to 1D and 2D point process datasets with small-to-moderate sizes.
Experimental setup
Datasets: Coal Mining Disaster Redwoods Porto Taxi synthetic stationary and nonstationary permanental process data
Benchmarks: Coal Mining Disaster Redwoods Porto Taxi synthetic stationary Gaussian-kernel data synthetic nonstationary data
Metrics: expected test log-likelihood RMSE runtime
Baselines: VBPP LBPP SSPP GSSPP GSSPP-M12 GSSPP-M52 DSSPP NSMPP
Builds on: sparse spectrum kernel permanental process Laplace approximation spectral mixture kernel deep spectral kernel
Closest papers by meaning
- Non-Stationary Spectral Kernels (NeurIPS 2017 · similarity 0.88)
- Revisiting Nonstationary Kernel Design for Multi-Output Gaussian Processes (ICLR 2026 · similarity 0.88)
- Sparse Spectrum Warped Input Measures for Nonstationary Kernel Learning (NeurIPS 2020 · similarity 0.88)
- Fourier Feature Approximations for Periodic Kernels in Time-Series Modelling (AAAI 2018 · similarity 0.88)
- A Generalized Stochastic Variational Bayesian Hyperparameter Learning Framework for Sparse Spectrum Gaussian Process Regression (AAAI 2017 · similarity 0.87)
- Sparse Learning of Dynamical Systems in RKHS: An Operator-Theoretic Approach (ICML 2023 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.