Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Stationarity without mean reversion in improper Gaussian processes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that improper GP priors with infinite variance and conditionally positive definite (non-positive) stationary kernels, such as Smooth Walk, Matérn Walk and Gaussian Walk, yield stationary but non-mean-reverting GP regression with analytic posterior corrections.
Problem
Stationary covariance functions used in GP regression are always mean reverting (for regular kernels), which causes pathological predictions on data that do not relax to a fixed global mean, such as stock prices or extrapolated trends.
Approach
Use non-positive (conditionally positive definite) stationary kernels derived from negative absolute value, such as -|tau| and -tau tanh(tau), combined with an infinite constant shift to form improper GP priors; the posterior has a closed-form correction of the usual GP formulas. Families of smooth kernels (Smooth Walk, Matérn Walk, Gaussian Walk) are built by convolving -|w| with proper stationar
Key results
- On S&P 500 stock forecasting, Gaussian Walk (improper) achieved NNL 3.90 and SEM 0.07, versus 76.02 for Squared Exponential (proper), per the stock table.
- On UCI datasets with 6 features and 100 subsampled points, GW (improper) achieved the best median NLL on 3droad (4.01) and concreteslump (9.37), while the proper SE GP got 96.02 and 1643.39 on those datasets respectively.
- The improper methods outperform proper GP and intrinsic kriging baselines on all but one of the UCI datasets in the low-dimensional table.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GP GW (improper) instead of the strongest listed baseline | GP regression · UCI 3droad (first 6 features) | median negative log-likelihood: 17.06 ± 1.18 -> 4.01 ± 0.16 | Helped |
| Use GP MW 1/2 (improper) instead of the strongest listed baseline | GP regression · UCI autompg (first 6 features) | median negative log-likelihood: 2.23 ± 0.23 -> 1.79 ± 0.04 | Helped |
| Use GP MW 1/2 (improper) instead of the strongest listed baseline | GP regression · UCI bike (first 6 features) | median negative log-likelihood: 1.40} \pm 0.03 -> 4.85 ± 0.3 | Hurt |
| Use GP GW (improper) instead of the strongest listed baseline | GP regression · UCI concreteslump (first 6 features) | median negative log-likelihood: 39.04 ± 2.87 -> 9.37 ± 0.66 | Helped |
| Use GP MW 1/2 (improper) instead of the strongest listed baseline | GP regression · UCI energy (first 6 features) | median negative log-likelihood: 2.75 ± 0.24 -> 2.41 ± 0.14 | Helped |
| Replace proper stationary kernels with improper conditionally positive definite kernels | GP regression · Stock prices | NNL: 76.02 -> 3.90 | Helped |
| Replace proper stationary kernels with improper conditionally positive definite kernels | GP regression · Stock prices | SEM: 21.19 -> 0.07 | Helped |
| Use improper Smooth Walk, Matérn Walk, and Gaussian Walk kernels rather than proper GP kernels | GP regression · UCI regression datasets (first 6 features) | median negative log-likelihood | Mixed |
Surprising findings
- The Hadamard finite part of the integral of 1/x^2 over [-1,1] is -2, so a positive integrand yields a negative regularized value, which is why these kernels are not positive definite.
- Removing the prior mean from an ordinary stationary covariance does not remove mean reversion, since the posterior mean still reverts to the data mean.
Limitations
- Validity of the isotropic extensions of Smooth Walk and Matérn Walk kernels in higher dimensions is only supported numerically, not proven analytically.
- Hyperparameters are optimized by marginal likelihood conditioned on a single data point because the true marginal likelihood diverges for improper models.
- Experiments use small subsamples (100 points) and only the first 6 features in the main UCI comparison.
Experimental setup
Datasets: S&P 500 daily closing stock prices (yfinance) UCI regression datasets (3droad, autompg, bike, concreteslump, energy, and 36 others in supplement) synthetic test functions sin(6t) - 6tanh(4t) and tanh(-x^2-y^2+5)
Benchmarks: UCI regression datasets S&P 500 stock price forecasting
Metrics: negative log-likelihood median negative log-likelihood standard error of the median (bootstrap) mean and median predictive negative log-likelihood
Baselines: Squared Exponential GP Matérn 1/2 GP Matérn 3/2 GP Matérn 5/2 GP Intrinsic Kriging (alpha=1) Intrinsic Kriging (alpha=0.5)
Builds on: Brownian motion kernel Matérn kernel intrinsic kriging vague basis function approach
Closest papers by meaning
- Non-Stationary Spectral Kernels (NeurIPS 2017 · similarity 0.86)
- Revisiting Nonstationary Kernel Design for Multi-Output Gaussian Processes (ICLR 2026 · similarity 0.85)
- Non-reversible Gaussian processes for identifying latent dynamical structure in neural data (NeurIPS 2020 · similarity 0.85)
- Robust and Conjugate Gaussian Process Regression (ICML 2024 · similarity 0.85)
- Fourier Feature Approximations for Periodic Kernels in Time-Series Modelling (AAAI 2018 · similarity 0.84)
- Robust and Conjugate Spatio-Temporal Gaussian Processes (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.