Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Energy-Efficient Gaussian Processes Using Low-Precision Arithmetic
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that Gaussian process regression can run with much lower-precision floating-point arithmetic (down to ~8 mantissa bits for well-conditioned kernels) via Cholesky or conjugate gradients, cutting circuit-level arithmetic energy by up to 89%.
Problem
Gaussian process regression is computed in double precision, which is energy-expensive; it is unknown what low-precision floating-point format keeps end-to-end model performance acceptable for typical small datasets.
Approach
Empirically map arbitrary low-precision floating-point formats (varying mantissa and exponent bits) for GPR inversion via Cholesky or modified conjugate gradients, relate the minimum precision to kernel-matrix condition number, and estimate energy savings from FPGA circuit-level benchmarks of adders and multipliers.
Key results
- Conjugate gradients with 8-bit mantissa on well-conditioned datasets gave up to 89.01% energy reduction for 97.99% of operations with test RMSE change within ±0.02.
- Cholesky decomposition achieved up to 88.94% energy reduction at 8 mantissa bits for well-conditioned data, covering 83.27%-87.43% of operations overall.
- Ill-conditioned California Housing needed at least 14 bits (conjugate gradients) or 24-34 bits (Cholesky) for competitive performance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use 8-bit rather than double-precision mantissas | Conjugate-gradient Gaussian process regression · Well-conditioned datasets, including Fried (fr) | ΔTest RMSE: 0.00 | No clear effect |
| Use 8-bit rather than double-precision mantissas | Conjugate-gradient Gaussian process regression · Fried (fr) | energy reduction: −89.01% | Helped |
| Use 14-bit rather than double-precision mantissas | Conjugate-gradient Gaussian process regression · California Housing (ch) | ΔTest RMSE: −0.02 | Helped |
| Reduce mantissa precision from 14 bits to 8 bits (ablation) | Conjugate-gradient Gaussian process regression · California Housing (ch) | ΔTest RMSE: −0.02 to 0.14 | Hurt |
| Reduce mantissa precision to 4 bits (ablation) | Conjugate-gradient Gaussian process regression · California Housing (ch) | ΔTest RMSE: 144.82 | Hurt |
| Use 8-bit rather than double-precision mantissas | Cholesky Gaussian process regression · Fried (fr) | ΔTest RMSE: −0.02 | No clear effect |
| Use 8-bit rather than double-precision mantissas | Cholesky Gaussian process regression · Fried (fr) | energy reduction: −88.94% | Helped |
| Use 34-bit rather than double-precision mantissas | Cholesky Gaussian process regression · California Housing (ch) | ΔTest RMSE: 0.01 | No clear effect |
| Reduce mantissa precision from 34 bits to 24 bits (ablation) | Cholesky Gaussian process regression · California Housing (ch) | ΔTest RMSE: 0.01 to 0.05 | Hurt |
| Reduce mantissa precision from 34 bits to 14 bits (ablation) | Cholesky Gaussian process regression · Pol (pl) | numerical stability | Mixed |
| Increase exponent bits from 11 to 15 at 24-bit mantissa (ablation) | Conjugate-gradient Gaussian process regression · California Housing (ch) | numerical stability: 4/5 stable to 4/5 stable | Helped |
| Increase exponent bits from 11 to 15 at 24-bit mantissa (ablation) | Cholesky Gaussian process regression · Pol (pl) | numerical stability: 24 bits unstable to stable | Helped |
Surprising findings
- Using larger exponent bits (15 instead of 11) stabilized computations that were unstable at higher mantissa precisions, suggesting exponent range rather than precision drove those instabilities.
- Some low-precision runs even slightly improved test RMSE, possibly via implicit regularization.
Limitations
- Energy savings come from simplified linear-complexity adder/multiplier circuits on an FPGA, not full system measurements; memory and bus energy are excluded.
- Evaluation restricted to small datasets (500 training points) and an RBF kernel.
Experimental setup
Datasets: Penn Machine Learning Benchmark datasets (fr, wn, st, mv, pl) California Housing
Benchmarks: Penn Machine Learning Benchmark (fr, wn, st, mv, pl) California Housing
Metrics: RMSE train RMSE test uncertainty calibration percent of arithmetic operations optimized energy reduction
Baselines: Cholesky decomposition in double precision conjugate gradients in double precision modified conjugate gradients (Maddox et al. 2022)
Builds on: Maddox et al. 2022 half-precision Gaussian processes Cholesky-Banachiewicz decomposition conjugate gradients
Closest papers by meaning
- Scalable Derivative Gaussian Processes via Exact Gradient Reduction (NeurIPS 2026 · similarity 0.83)
- KernelMatmul: Scaling Gaussian Processes to Large Time Series (AAAI 2025 · similarity 0.83)
- Towards Learning High-Precision Least Squares Algorithms with Sequence Models (ICLR 2025 · similarity 0.83)
- Computation-Aware Gaussian Processes: Model Selection And Linear-Time Inference (NeurIPS 2024 · similarity 0.83)
- Improving Linear System Solvers for Hyperparameter Optimisation in Iterative Gaussian Processes (NeurIPS 2024 · similarity 0.83)
- Turbocharging Gaussian Process Inference with Approximate Sketch-and-Project (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.