Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Deep Random Features for Scalable Interpolation of Spatiotemporal Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes deep random features, neural networks whose layers are random Fourier (or spherical Gegenbauer) feature expansions of stationary GPs, for scalable spatiotemporal interpolation of remote-sensing data with ensemble-based uncertainty.
Problem
Scalable, expressive interpolation of sparse spatiotemporal earth observation data (planar and global/spherical) with calibrated uncertainty, where GPs need inducing-point approximations and deep GPs need crude variational inference.
Approach
Build deep neural networks from random feature layers that approximate stationary GPs on the plane and sphere (cosine features and Gegenbauer-polynomial features for S^2), train only the output weights with mini-batched gradient descent, and quantify uncertainty via deep ensembles, VI, or dropout; hyperparameters tuned by Bayesian optimisation on validation loss with functional regularisation.
Key results
- On synthetic MSS data, DRF (Ensembles) achieved the best CRPS (0.046) among methods, comparable to GPSat (0.045) and with RMSE 0.135 vs GPSat 0.126.
- On sea-ice freeboard, DRF ensembles achieved CRPS 0.077, matching the best (tied with GPSat at 0.076 within noise on CRPS ordering, and SVGP NLPD -1.300 was best), with RMAE 0.322 vs FFN's 0.316.
- DRF ensembles trained in about 19.7 minutes vs 63.6 for GPSat and 42.3 for DGP on the synthetic experiment.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use deep ensembles instead of variational inference for uncertainty quantification (ablation) | DRF · Synthetic MSS interpolation | CRPS: 0.071 ± 0.019 -> 0.046 ± 0.005 | Helped |
| Use variational inference instead of deep ensembles for uncertainty quantification (ablation) | DRF · Synthetic MSS interpolation | NLL: 13.590± 4.899 -> -0.407± 0.756 | Helped |
| Use test-time dropout instead of deep ensembles for uncertainty quantification (ablation) | DRF · Synthetic MSS interpolation | RMSE: 0.135± 0.006 -> 0.238± 0.001 | Hurt |
| Increase depth from one layer to four layers (ablation) | DRF · Synthetic MSS interpolation | RMSE | Helped |
| Increase depth beyond four layers to 10 or 20 layers (ablation) | DRF · Synthetic MSS interpolation | RMSE | Hurt |
| Add functional regularisation during hyperparameter search (ablation) | DRF · Sea-ice freeboard interpolation | RMAE | Helped |
| Use spherical random features rather than Euclidean coordinates for the spatial input (ablation) | DRF · Global sea level anomaly interpolation | fine-scale fluctuation capture | Helped |
| Use Huber loss instead of MSE (ablation) | Spherical DRF · Global sea level anomaly interpolation | robustness to outliers | Helped |
| Use random spherical features instead of deterministic spherical features (ablation) | Spherical DRF · Spherical interpolation | numerical stability | Helped |
| Use DRF ensembles rather than GPSat | DRF · Synthetic MSS interpolation | RMSE: 0.126± 0.001 -> 0.135± 0.006 | Hurt |
| Use DRF ensembles rather than GPSat | DRF · Sea-ice freeboard interpolation | RMAE: 0.318± 0.000 -> 0.322± 0.000 | Hurt |
Surprising findings
- Variational inference gave the best NLL on synthetic data despite underconfident predictions, while ensembles gave better CRPS and RMSE.
- Functional regularisation was needed during hyperparameter search on real noisy altimetry data, since validation-only tuning overfit.
Limitations
- Kernel hyperparameter tuning via Bayesian optimisation is difficult and requires knowledge of hyperparameter ranges; hyperparameters are less interpretable in deep architectures.
- Functional regularisation penalty weight alpha must be hand-tuned, adding a hyper-hyperparameter.
- Euclidean DRF on the globe shows distortions near poles and a discontinuity at longitude 0 due to stereographic projection.
Experimental setup
Datasets: Sentinel-3A/3B and CryoSat-2 altimetry-derived mean sea surface height (synthetic) Sentinel-3A/3B and CryoSat-2 sea-ice freeboard altimetry Sentinel-3A/3B sea level anomaly
Benchmarks: synthetic MSS interpolation sea-ice freeboard interpolation global sea level anomaly interpolation
Metrics: RMSE RMAE NLL NLPD CRPS training time
Baselines: SVGP DGP (doubly stochastic VI) GPSat ReLU MLP Fourier features network (FFN) SIREN conditional neural process
Builds on: deep Gaussian processes random Fourier features deep ensembles Matérn Gaussian processes on the sphere sparse variational GP
Closest papers by meaning
- Inter-domain Deep Gaussian Processes (ICML 2020 · similarity 0.87)
- Decoupled Spatiotemporal Forecasting from Extreme Sparse Observations via Quantized Latent Space (AAAI 2026 · similarity 0.86)
- Causal Deciphering and Inpainting in Spatio-Temporal Dynamics via Diffusion Model (NeurIPS 2024 · similarity 0.86)
- Spatio-Temporal Variational Gaussian Processes (NeurIPS 2021 · similarity 0.86)
- Continuous Field Reconstruction from Sparse Observations with Implicit Neural Networks (ICLR 2024 · similarity 0.85)
- Learning Spatiotemporal Dynamical Systems from Point Process Observations (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.