Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Uncertainty Quantification for Data-Driven Change-Point Learning via Cross-Validation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Controls the probability of overestimating the number of change-points selected by cross-validation, by sequential high-dimensional multiplier-bootstrap tests of model-comparison discrepancies with an order-preserved odd/even sample split.
Problem
Cross-validation for selecting the number of change-points tends to slightly overestimate in finite samples, and existing information-criterion or penalty-based methods are model-dependent; there is no uncertainty quantification for overestimation.
Approach
Frame the choice of the number of change-points as a sequence of hypothesis tests comparing prediction-error discrepancies between r and larger models (H0: max over s>r of Delta_{r,s} <= 0), using a high-dimensional multiplier bootstrap critical value; define U = K_CV - K_min as an uncertainty margin so that P(K_CV - K_n > U) is bounded by alpha, with a V-fold refinement.
Key results
- On synthetic Gaussian data with n=1000 and SNR 1.2, SS-PELT keeps P+ at or below nominal alpha (e.g., 1.0% at alpha=10%, K_n=25), while SMUCE shows P+ of 63.5-83% under t(5) errors.
- Under t(5) errors with alpha=5%, SMUCE P+ reaches 63.5% for K_n=25 while SS-PELT achieves 5.5%.
- On the real CGH data, CV-WBS gives K_hat - K_n = 1 versus 18 for SMUCE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SS-PELT instead of SMUCE | Change-point uncertainty quantification · Synthetic t(5), K_n=25, α=10% | P_+: 77.5 -> 9.0 | Helped |
| Use SS-PELT instead of SMUCE | Change-point uncertainty quantification · Synthetic t(5), K_n=35, α=10% | P_+: 71.0 -> 11.5 | Helped |
| Use SS-PELT instead of SMUCE | Change-point uncertainty quantification · Synthetic Gaussian, K_n=35, α=10% | K_hat - K_n: -0.1 -> 0.5 | Mixed |
| Add three-fold cross-validation refinement to sample splitting (ablation) | PELT uncertainty quantification · Synthetic Gaussian, SNR=0.9, K_n=15, α=10% | P_+: 2.5 -> 0.0 | Helped |
| Add three-fold cross-validation refinement to sample splitting (ablation) | PELT uncertainty quantification · Synthetic Gaussian, SNR=0.9, K_n=25, α=10% | P_+: 2.5 -> 0.0 | Helped |
| Add three-fold cross-validation refinement to sample splitting (ablation) | PELT uncertainty quantification · Synthetic Gaussian, SNR=1.2, K_n=15, α=10% | P_+: 1.5 -> 0.0 | Helped |
| Add three-fold cross-validation refinement to sample splitting (ablation) | PELT uncertainty quantification · Synthetic Gaussian, SNR=1.2, K_n=25, α=10% | P_+: 1.0 -> 0.0 | Helped |
| Use CV-WBS instead of SMUCE | Change-point uncertainty quantification · Array CGH, GM05296 | K_hat - K_n: 18 -> 1 | Helped |
Surprising findings
- SMUCE exceeds the nominal overestimation level by a wide margin under heavy-tailed t(5) errors (P+ up to 83%), while it is well controlled under Gaussian errors.
Limitations
- Assumes parametric, low-dimensional change-point models and independent observations.
- Theoretical guarantees rely on moment and high-dimensional CLT conditions and a condition on the overfitting regime.
- Real-data evaluation uses a single cell line with an elbow-based reference for K_n.
Experimental setup
Datasets: coriell array CGH data (DNAcopy R package, cell line GM05296)
Benchmarks: synthetic mean-shift change-point simulations coriell array CGH
Metrics: empirical overestimation probability P+ average K_CV - K_n average U
Baselines: SMUCE CV with WBS CV with PELT
Builds on: cross-validation with confidence SMUCE wild binary segmentation PELT
Closest papers by meaning
- Crossing the Validation crisis: Cross-validation reduces benchmarking variance surprisingly well (NeurIPS 2026 · similarity 0.85)
- Fixed-Confidence Multiple Change Point Identification under Bandit Feedback (ICML 2025 · similarity 0.84)
- Efficient Analytic Uncertainty Quantification for Multimodal Regression (NeurIPS 2026 · similarity 0.84)
- Data Mixture Optimization: A Multi-fidelity Multi-scale Bayesian Framework (NeurIPS 2025 · similarity 0.83)
- The Sample Complexity of Multiple Change Point Identification under Bandit Feedback (NeurIPS 2026 · similarity 0.83)
- Quantifying Uncertainty in the Presence of Distribution Shifts (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.