Research map / Probabilistic and scientific modeling / Molecules and proteins
FAB-PPI: Frequentist, Assisted by Bayes, Prediction-Powered Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FAB-PPI, which places a horseshoe (or Gaussian) prior on the PPI rectifier and builds frequentist-valid confidence regions via the FAB framework, giving shorter intervals when predictions are good and reverting to standard PPI when they are poor.
Problem
Prediction-powered inference (PPI) combines few labelled points with many ML predictions for valid inference, but its rectifier estimation ignores prior knowledge that predictions are usually good with occasional large errors, so intervals may be wider than necessary.
Approach
Encode prior beliefs about prediction quality as a horseshoe prior on the rectifier and apply Frequentist-Assisted-by-Bayes (FAB) confidence regions to it, yielding intervals that are shorter when the rectifier is near zero, keep exact frequentist coverage for any prior, and revert to standard PPI for large rectifiers.
Key results
- On synthetic biased predictions, FAB-PPI with horseshoe prior gives smaller CIs than PPI when bias is small and reverts to PPI-sized intervals for large bias, while the Gaussian prior's intervals grow without bound.
- On real datasets with high-quality predictions (galaxies, forest, AlphaFold, healthcare), FAB-PPI/FAB-PPI++ outperform classical inference and corresponding PPI methods in MSE and CI volume with comparable coverage.
- Under the horseshoe prior the FAB-PPI estimator is consistent whereas the Gaussian-prior version is not, per the paper's proposition.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add horseshoe-prior adaptive shrinkage to PPI | PPI on noisy-prediction mean estimation · Synthetic noisy predictions | CI volume | Helped |
| add power tuning to FAB-PPI | FAB-PPI on noisy-prediction mean estimation · Synthetic noisy predictions | CI volume | Helped |
| add horseshoe-prior adaptive shrinkage to PPI++ | PPI++ on mean estimation · Synthetic biased predictions | CI volume | Helped |
| use a horseshoe rather than Gaussian prior | FAB-PPI on biased-prediction mean estimation · Synthetic biased predictions | CI volume | Helped |
| add FAB-PPI adaptive shrinkage | PPI on real-data estimation · Galaxies, Forest, AlphaFold, and Healthcare | MSE | Helped |
| add FAB-PPI adaptive shrinkage | PPI on real-data estimation · Galaxies, Forest, AlphaFold, and Healthcare | CI volume | Helped |
| use a horseshoe rather than Gaussian prior | FAB-PPI on quantile estimation · Genes | MSE | Helped |
| use a horseshoe rather than Gaussian prior | FAB-PPI on quantile estimation · Genes | CI volume | Helped |
| add FAB-PPI shrinkage | PPI on linear regression · Census, parameter in panel (a) | performance | Hurt |
| use a horseshoe rather than Gaussian prior | FAB-PPI on linear regression · Census, parameter in panel (a) | performance | Helped |
Surprising findings
- The Gaussian-prior FAB-PPI estimator is inconsistent even when the PPI estimator is consistent.
- On the census regression, FAB-PPI with horseshoe underperforms the alternatives for small n before matching PPI as n grows.
Limitations
- For general problems beyond mean estimation and linear regression, the method requires evaluating the estimating equation over a grid of theta, which is computationally expensive in high dimensions.
- Experiments assume N large enough to ignore uncertainty in the measure-of-fit term m_theta, simplifying the CI construction.
- Horseshoe prior with a rescaled noise parameter eta != 1 requires numerical integration, increasing cost.
Experimental setup
Datasets: AlphaFold galaxies forest healthcare census genes
Benchmarks: synthetic mean estimation (biased and noisy predictions) AlphaFold odds ratio galaxies mean estimation forest mean estimation healthcare logistic regression genes quantile estimation census linear regression
Metrics: MSE CI volume CI coverage
Baselines: classical inference PPI PPI++ Gaussian-prior FAB-PPI
Builds on: prediction-powered inference PPI++ frequentist-assisted by Bayes (FAB) horseshoe prior control variates
Closest papers by meaning
- Anytime-valid, Bayes-assisted, Prediction-Powered Inference (NeurIPS 2025 · similarity 0.90)
- Prediction-Powered Adaptive Shrinkage Estimation (ICML 2025 · similarity 0.89)
- Regression for the Mean: Auto-Evaluation and Inference with Few Labels through Post-hoc Regression (ICML 2025 · similarity 0.86)
- Prediction-Powered Semi-Supervised Learning with Online Power Tuning (NeurIPS 2025 · similarity 0.85)
- Stratified Prediction-Powered Inference for Effective Hybrid Evaluation of Language Models (NeurIPS 2024 · similarity 0.85)
- Synthetic-powered predictive inference (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.