Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Bayesian leave-one-out cross-validation for large data
Måns Magnusson, Michael Andersen, Johan Jonasson, Aki Vehtari
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace simple random sampling with PPS proportional to -log predictive density | LOO estimation on LR(c) 100D, m=10 · LR(c) 100D | SE(elpd_loo estimate): 2197.8 -> 11.8 | Improved |
| Replace simple random sampling with PPS proportional to -log predictive density | LOO estimation on LR(c) 100D, m=100 · LR(c) 100D | SE(elpd_loo estimate): 598.8 -> 4.3 | Improved |
| Replace simple random sampling with PPS proportional to -log predictive density | LOO estimation on LR(c) 10D, m=10 · LR(c) 10D | SE(elpd_loo estimate): 3310.2 -> 3.2 | Improved |
| Use point-estimate likelihood rather than full-posterior predictive density to set PPS probabilities (ablation) | PPS LOO estimation · LR(c) 100D, m=100 | SE(elpd_loo estimate): 4.3 -> 4.5 | No clear effect |
| Use point-estimate likelihood rather than full-posterior predictive density to set PPS probabilities (ablation) | PPS LOO estimation · LR(c) 100D, m=1000 | SE(elpd_loo estimate): 1.5 -> 1.5 | No clear effect |
| Use point-estimate likelihood rather than full-posterior predictive density to set PPS probabilities (ablation) | PPS LOO estimation · LR(c) 10D, m=100 | SE(elpd_loo estimate): 1.2 -> 1.1 | No clear effect |
| Use mean-field variational inference for a correlated posterior (ablation) | Bayesian linear regression · LR(c) 100D | elpd_loo: -14247 -> -14267 | No clear effect |
| Use full-rank ADVI at high parameter dimension (ablation) | Bayesian linear regression · LR(i) 100D | elpd_loo: -14239 -> -14193 | Mixed |
| Increase PPS subsample size from 10 to 100 (ablation) | PPS(1) LOO estimation · LR(c) 100D | SE(elpd_loo estimate): 11.8 -> 4.3 | Improved |
| Increase PPS subsample size from 100 to 1000 (ablation) | PPS(1) LOO estimation · LR(c) 100D | SE(elpd_loo estimate): 4.3 -> 1.5 | Improved |
| Use ADVI(MF) rather than MCMC for Radon model 6 (ablation) | Radon model 6 with PPS subsample m=500 · Radon | elpd_loo estimate: -16835 -> -16656 | Mixed |
| Use PPS sampling instead of simple random sampling as n grows | LOO estimation with point-estimate likelihood probabilities · LR(c) 100D, m=10, n=100000 | SE(elpd_loo estimate): 19721 -> 17.5 | Improved |
Closest papers by meaning
- Is Cross-validation the Gold Standard to Estimate Out-of-sample Model Performance? (NeurIPS 2024 · similarity 0.83)
- Cross-Validated Off-Policy Evaluation (AAAI 2025 · similarity 0.81)
- Hypothesis Testing for Generalized Thurstone Models (ICML 2025 · similarity 0.81)
- The Silent Hyperparameter: Quantifying the Impact of Inference Backends on LLM Reproducibility (NeurIPS 2026 · similarity 0.81)
- One Sample Fits All: Approximating All Probabilistic Values Simultaneously and Efficiently (NeurIPS 2024 · similarity 0.81)
- Crossing the Validation crisis: Cross-validation reduces benchmarking variance surprisingly well (NeurIPS 2026 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
