Research map / Learning theory and algorithms / Kernels and regression theory
Inducing Clusters Deep Kernel Gaussian Process for Longitudinal Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ICDKGP, a zero-mean deep kernel GP with a state-space non-zero mean function and inducing clusters (cluster centers as inducing points) for irregular longitudinal regression with abrupt discontinuities.
Problem
Predictive modeling from irregularly and sparsely sampled longitudinal data with unknown complex correlation structures and abrupt discontinuities in outcomes.
Approach
Combines a longitudinal deep kernel GP (zero-mean) with a deterministic state-space mean function to capture discontinuities, and replaces inducing points with inducing clusters tied to training-data cluster centers, relaxed into a soft proximity-based constrained optimization with a bounded-error argument.
Key results
- On simulated multi-level correlation data with smooth targets, ICDKGP reaches 99.5±0.5 R2 for MC(C=2), while L-DKGPR gets 91.3±0.2 and SVGP -102.7±83.1.
- On full TADPOLE, ICDKGP achieves 63.1±2 R2 versus 64.9±1 for L-DKGPR, so the proposed model does not always win on large data.
- ICDKGP outperforms L-DKGPR on small real datasets (e.g., TADPOLES 53.8 vs 44.0, GSSS 25.3 vs 19.1), with the gap narrowing as N grows.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ICDKGP instead of L-DKGPR | Longitudinal regression · Simulated smooth LC | R2 (%): 84.2±2.9 vs 86.0±0.2 | Hurt |
| Use ICDKGP instead of L-DKGPR | Longitudinal regression · Simulated smooth MC (C = 2) | R2 (%): 91.3±0.2 -> 99.5±0.5 | Helped |
| Use ICDKGP instead of L-DKGPR | Longitudinal regression · Simulated non-smooth LC | R2 (%): 76.8±17.8 -> 84.2±5.2 | Helped |
| Use ICDKGP instead of L-DKGPR | Longitudinal regression · Simulated non-smooth MC (C = 5) | R2 (%): 83.4±7.8 -> 93.1±3.2 | Helped |
| Use ICDKGP instead of the strongest listed baseline | Longitudinal regression · TADPOLES | R2 (%): 50.8±6 -> 53.8±5 | Helped |
| Use ICDKGP instead of GEE | Longitudinal regression · SWANS | R2 (%): 46.4±8 -> 47.9±4 | Helped |
| Use ICDKGP instead of L-DKGPR | Longitudinal regression · GSSS | R2 (%): 19.1±4 -> 25.3±3 | Helped |
| Use ICDKGP instead of L-DKGPR | Longitudinal regression · TADPOLEL | R2 (%): 64.9±1 -> 63.1±2 | Hurt |
| Use ICDKGP instead of L-DKGPR | Longitudinal regression · SWANL | R2 (%): 52.5±0 -> 54.2±0 | Helped |
| Use ICDKGP instead of L-DKGPR | Longitudinal regression · GSSL | R2 (%): 56.9±0 -> 56.4±1 | Hurt |
| Add inducing clusters instead of standard inducing points (ablation) | DKGP · TADPOLEL | R2 (%): 60.7±3.9 -> 63.1±1.8 | Helped |
| Add a state-space mean function (ablation) | DKGP-ZM · SWANL | R2 (%): 43.3±3.8 -> 54.4±0.5 | Helped |
Surprising findings
- On the full, large TADPOLE dataset, L-DKGPR (64.9) slightly outperforms ICDKGP (63.1), so the advantage is mainly on smaller datasets.
- Zero-mean DKGP-ZM needs roughly an order of magnitude more inducing points (M=100) to approach the predictive performance of the non-zero-mean DKGP with M=10.
Limitations
- Advantage over L-DKGPR narrows or reverses on large datasets (e.g., full TADPOLE).
- Theoretical equivalence relies on assumptions such as apparent mixture-of-Gaussian structure in the latent space, and the constraint hyperparameters must be sufficiently small.
- Training is MLE/ELBO based and may get stuck in local maxima of the kernel search.
Experimental setup
Datasets: SWAN GSS TADPOLE simulated longitudinal data
Benchmarks: simulated longitudinal data (smooth and non-smooth) SWAN GSS TADPOLE
Metrics: R2 (%) correlation structure recovery
Baselines: GLMM GEE LMLFM L-DKGPR SVGP DSVGP SKIPGP DKGP DKGP-ZM
Builds on: Liang et al. 2021 longitudinal deep kernel GP (L-DKGPR) sparse variational GP (Titsias 2009, Hensman et al. 2015) deep kernel learning Wilson et al. 2016 deep kernel ELBO
Closest papers by meaning
- Longitudinal Deep Kernel Gaussian Process Regression (AAAI 2021 · similarity 0.91)
- gp2Scale: A Class of Compactly Supported Non-Stationary Kernels and Distributed Computing for Exact Gaussian Processes on 10 Million Data Points (ICML 2026 · similarity 0.86)
- Deep Gaussian Processes for Regression using Approximate Expectation Propagation (ICML 2016 · similarity 0.85)
- Scalable Multiple Kernel Clustering: Learning Clustering Structure from Expectation (ICML 2024 · similarity 0.85)
- KernelMatmul: Scaling Gaussian Processes to Large Time Series (AAAI 2025 · similarity 0.84)
- Convolutional Deep Kernel Machines (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.