Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes

Sparse Variational Student-t Processes

AAAI 2024 · Probabilistic and scientific modeling › Uncertainty and Gaussian processes · approach: Sparse variational Gaussian processes · 5 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Sparse Variational Student-t Processes (SVTP), a sparse inducing-point variational framework for Student-t processes with two lower-bound variants (SVTP-MC and SVTP-UB) for regression on heavy-tailed or outlier-laden data.

Problem

Full Student-t processes scale as O(n^3) like Gaussian processes, and sparse methods have been little developed for them, so their use on large heavy-tailed or outlier-contaminated regression datasets is limited.

Approach

Uses the closed-form conditional of the multivariate Student-t to define inducing points for a TP prior, then optimizes a variational ELBO with a reparameterized Student-t variational posterior; the KL term is estimated either by Monte Carlo (SVTP-MC) or with a Jensen upper bound (SVTP-UB). Claims the log-based likelihood term makes it more robust to outliers than SVGP.

Key results

• On Yacht, SVTP+UB achieves MSE 1.44 vs 7.56 for SVGP, and on Concrete_Outliers SVTP+MC reaches MSE 6.40 vs 12.20 for SVGP.
• Table 1 timing shows SVTP-UB with m=n/10 takes 0.931s per epoch on Elevator vs 97.977s for full TP, while full TP fails on Protein and Taxi.
• Table 3 reports SVGP KL-term values of 56 (UB) vs 2.2 (MC) on Boston, suggesting the UB estimate is much looser for larger datasets.

Measured effects reported in the paper

Surprising findings

• SVTP-MC outperforms SVTP-UB on most large datasets (e.g., Taxi, Kin8nm), contrary to the expectation that the tighter bound would help most in the small-data regime.
• The reported KL term values in Table 3 are labeled SVGP+UB/SVGP+MC, which does not clearly match the SVTP method being analyzed.

Limitations

• Robustness-to-outliers argument is a heuristic comparison of ELBO terms rather than a formal result
• Evaluation only covers regression with a squared exponential kernel and fixed noise 0.10
• Method is restricted to regression; classification and multi-output tasks are left to future work

Experimental setup

Datasets: Concrete Boston Kin8nm Yacht Energy Elevator Protein Taxi Trip Fare

Benchmarks: Concrete Boston Kin8nm Yacht Energy Elevator Protein Taxi Trip Fare Concrete_Outliers Kin8nm_Outliers

Metrics: MSE test log likelihood time per epoch KL term value

Baselines: SVGP full Student-t process (TP)

Builds on: sparse variational Gaussian processes Student-t process reparameterization trick

Closest papers by meaning

• A Generalized Stochastic Variational Bayesian Hyperparameter Learning Framework for Sparse Spectrum Gaussian Process Regression (AAAI 2017 · similarity 0.90)
• Variational Inference for Gaussian Process Models with Linear Complexity (NeurIPS 2017 · similarity 0.90)
• New Bounds for Sparse Variational Gaussian Processes (ICML 2025 · similarity 0.90)
• Scalable Variational Bayesian Kernel Selection for Sparse Gaussian Process Regression (AAAI 2020 · similarity 0.89)
• Robust Gaussian Processes via Relevance Pursuit (NeurIPS 2024 · similarity 0.87)
• Amortized Structured Stochastic Variational Inference for Gaussian Process Latent Variable Models (NeurIPS 2026 · similarity 0.87)