Research map / Learning theory and algorithms / Kernels and regression theory
Scalable and adaptive prediction bands with kernel sum-of-squares
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds scalable, adaptive split-conformal prediction bands by learning a kernel sum-of-squares scaling function via a representer theorem and an accelerated-gradient dual, and tunes its lengthscale with an HSIC-based local-coverage criterion.
Problem
Standard split conformal prediction yields constant-width bands, lacking adaptivity to local variability, and existing kernel sum-of-squares approaches for learning adaptive bands rely on SDP solvers that scale only to a few hundred samples; tuning kernel lengthscales for test-conditional coverage i
Approach
Learn the rescaling function of the conformal score as a non-negative kernel sum-of-squares, derive a representer theorem and a dual with only n+1 variables solved by accelerated gradient ascent, and tune the scaling-function lengthscale by maximizing HSIC between residuals and the scale function (a bound on local coverage).
Key results
- On test case 1 (n=100), kernel SoS with HSIC-tuned lengthscale gives smaller mean width than CQR and rescaled GPs while maintaining satisfying local coverage.
- The dual solver scales to n=2000 training points, whereas the SDP solver (SCS) is limited to about n=200.
- HSIC discriminates the lengthscale more stably than mutual information, which the paper reports as numerically unusable in practice.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Tune the kernel lengthscale with HSIC rather than MI | Kernel SoS · Test case 2, n=100 | lengthscale-selection stability | Helped |
| Use HSIC-optimized lengthscale for adaptive score scaling | Kernel SoS · Test case 1, n=100 | mean interval width | Helped |
| Use Kernel SoS instead of heteroscedastic GP | Split conformal prediction · Test case 1, n=100 | mean interval width | Helped |
| Use Kernel SoS instead of heteroscedastic GP | Split conformal prediction · Test case 2, n=100 | mean interval width | Helped |
| Use Kernel SoS instead of homoscedastic GP | Split conformal prediction · Test case 3, n=100 | local coverage | Helped |
| Use Kernel SoS instead of GP methods | Split conformal prediction · Test case 4, n=100 | mean interval width | Hurt |
| Use the dual solver instead of an off-the-shelf SDP solver | Kernel SoS · Test case 1, large-scale training | scalable training size: SDP: n=200; dual: n=2000 | Helped |
| Use an overly small kernel lengthscale instead of the HSIC-optimized lengthscale (ablation) | Kernel SoS · Test case 2, n=100 | adaptivity | Hurt |
| Set a=0 in the symmetric-noise setting (ablation) | Kernel SoS · Symmetric-noise test cases | RMSE | No clear effect |
| Vary λ₂ while keeping λ₂ positive (ablation) | Kernel SoS · Hyperparameter study | mean interval width | No clear effect |
| Set λ₁=1 (ablation) | Kernel SoS · Hyperparameter study | solution shape | No clear effect |
| Increase b (ablation) | Kernel SoS · Hyperparameter study | mean interval width | Helped |
Surprising findings
- The mean-squared-error term weight a has little influence on RMSE under symmetric noise, so the authors set a=0 in that setting.
- The lambda_1 regularization has little impact on the solution once positive, and lambda_2 has a very small impact on interval shape as long as it is positive.
Limitations
- Requires lambda_2 > 0 for the dual formulation, excluding some prior frameworks.
- Coverage constraints at 100% on pre-training data may make the method sensitive to outliers.
- Experiments are on small synthetic test cases; the HSIC criterion is only shown for kernel SoS and not generalized to other score functions.
Experimental setup
Datasets: synthetic test cases 1-4
Benchmarks: synthetic test cases 1-4
Metrics: mean interval width mutual information R2_SQI local coverage RMSE HSIC solve time
Baselines: CQR rescaled GP (homoscedastic) heteroscedastic GP SDP solver (SCS)
Builds on: kernel sum-of-squares conformalized quantile regression split conformal prediction rlcp
Closest papers by meaning
- Optimal transport-based conformal prediction (ICML 2025 · similarity 0.89)
- A Kernel Nonconformity Score for Multivariate Conformal Prediction (NeurIPS 2026 · similarity 0.88)
- Co-optimization for Adaptive Conformal Prediction (NeurIPS 2026 · similarity 0.88)
- Kernel-based Optimally Weighted Conformal Time-Series Prediction (ICLR 2025 · similarity 0.88)
- Backward Conformal Prediction (NeurIPS 2025 · similarity 0.88)
- Conformal Prediction via Regression-as-Classification (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.