Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Orthogonal Bootstrap: Efficient Simulation of Input Uncertainty
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Orthogonal Bootstrap, which splits the bootstrap simulation target into a closed-form influence-function (infinitesimal jackknife) part and a simulated orthogonal remainder, reducing the Monte Carlo replications needed from Omega(n) to O(1) for debiasing and variance/CI estimation.
Problem
Bootstrap simulation of input uncertainty is computationally expensive because the number of Monte Carlo replications needed scales with data size, and small-B bootstrap gives high simulation error and poor coverage.
Approach
Use the influence function (infinitesimal jackknife, closed form with mean zero) as a control variate for the bootstrap output, so only the orthogonal residual must be simulated; this keeps the bootstrap expectation while cutting simulation variance from O(n^-1) to O(n^-2) per replication.
Key results
- In the variance-of-folded-normal CI experiment with B=2, Orthogonal Bootstrap reaches 0.952 coverage with width 0.076 versus 0.591 coverage for Standard Bootstrap at similar width (0.043).
- Cheap Bootstrap at B=2 achieves 0.956 coverage but roughly doubles widths (e.g., 0.145 vs 0.076 for folded normal variance), while Orthogonal Bootstrap keeps widths matching the standard bootstrap.
- Theory: Standard Bootstrap needs Omega(n) Monte Carlo replications versus O(1) for Orthogonal Bootstrap to achieve the required simulation error.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Orthogonal Bootstrap instead of Standard Bootstrap with B=2 | variance confidence intervals · Variance of Folded normal | coverage: 0.591 -> 0.952 | Helped |
| Use Orthogonal Bootstrap instead of Standard Bootstrap with B=2 | variance confidence intervals · Variance of Double exponential | coverage: 0.580 -> 0.949 | Helped |
| Use Orthogonal Bootstrap instead of Standard Bootstrap with B=2 | confidence intervals · Correlation of Bivariate Lognormal | coverage: 0.566 -> 0.911 | Helped |
| Use Orthogonal Bootstrap instead of Standard Bootstrap with B=2 | confidence intervals · Linear Regression | coverage: 0.632 -> 0.946 | Helped |
| Use Orthogonal Bootstrap instead of Cheap Bootstrap with B=2 | variance confidence intervals · Variance of Folded normal | width (st. dev.): 0.145(0.077) -> 0.076(0.008) | Helped |
| Use Orthogonal Bootstrap instead of Cheap Bootstrap with B=2 | variance confidence intervals · Variance of Double exponential | width (st. dev.): 1.085(0.606) -> 0.552(0.077) | Helped |
| Use Orthogonal Bootstrap instead of Cheap Bootstrap with B=2 | confidence intervals · Correlation of Bivariate Lognormal | width (st. dev.): 0.385(0.239) -> 0.194(0.063) | Helped |
| Use Orthogonal Bootstrap instead of Cheap Bootstrap with B=2 | confidence intervals · Linear Regression | width (st. dev.): 1.199(0.615) -> 0.615(0.076) | Helped |
| Use Orthogonal Bootstrap instead of Standard Bootstrap with B=2 | prediction intervals · Yacht Hydrodynamics | coverage: 0.86 -> 0.99 | Helped |
| Use Orthogonal Bootstrap instead of Cheap Bootstrap with B=2 | prediction intervals · Yacht Hydrodynamics | width: 82.26 -> 45.88 | Helped |
| Use Orthogonal Bootstrap instead of Standard Bootstrap with B=2 | prediction intervals · Energy Efficiency | coverage: 0.932 -> 0.951 | Helped |
| Use Orthogonal Bootstrap instead of Cheap Bootstrap with B=2 | prediction intervals · Energy Efficiency | width: 36.03 -> 15.54 | Helped |
Surprising findings
- The Infinitesimal Jackknife alone gives a much wider interval (1.679) than the bootstrap for linear regression, while Orthogonal Bootstrap matches the bootstrap width (0.623).
Limitations
- Variance estimator can be negative for small sample size, requiring an improved estimator.
- Theory relies on smoothness assumptions (C^1 Frechet derivative under kernel MMD, moment conditions).
- Requires computing influence functions, e.g., via inverse Hessian-vector products or KKT differentiation, which can be costly.
Experimental setup
Datasets: UCI Yacht Hydrodynamics UCI Energy Efficiency UCI Kin8nm simulated distributions (folded normal, double exponential, bivariate lognormal, linear regression)
Benchmarks: UCI Yacht Hydrodynamics UCI Energy Efficiency UCI Kin8nm simulated debiasing examples simulated confidence interval examples
Metrics: bias coverage interval width
Baselines: Standard Bootstrap Cheap Bootstrap Infinitesimal Jackknife
Builds on: Cheap Bootstrap Infinitesimal Jackknife double/orthogonal machine learning
Closest papers by meaning
- Bootstrap in High Dimension with Low Computation (ICML 2023 · similarity 0.87)
- Centroid Approximation for Bootstrap: Improving Particle Quality at Inference (ICML 2022 · similarity 0.87)
- Statistical inference for Linear Stochastic Approximation with Markovian Noise (NeurIPS 2025 · similarity 0.82)
- Uncertainty Quantification with the Empirical Neural Tangent Kernel (NeurIPS 2025 · similarity 0.82)
- Integer Programming for Generalized Causal Bootstrap Designs (ICML 2025 · similarity 0.81)
- Gaussian Approximation and Multiplier Bootstrap for Polyak-Ruppert Averaged Linear Stochastic Approximation with Applications to TD Learning (NeurIPS 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.