Research map / Probabilistic and scientific modeling / Variational and generative sampling
Empirical Bayes Rebiasing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes empirical Bayes rebiasing, which estimates the bias distribution across many tasks (NPMLE or normal) and partially undoes full debiasing to produce shorter calibrated intervals for PPI and family-based GWAS.
Problem
Constructing short calibrated confidence intervals for many parameters when each biased estimate has a noisy unbiased bias estimate; full debiasing inflates variance and yields long intervals.
Approach
Model task-level biases as drawn from an unknown distribution G learned across tasks (normal or NPMLE), then subtract the posterior mean of the bias instead of the full bias estimate, yielding a rebiased estimator and intervals that interpolate between the biased and debiased estimators; coverage is treated in a partially Bayes framework with theta_i fixed.
Key results
- On LM Arena (n=298), RB intervals at 90% nominal coverage are about 23% shorter than Classical and 19% shorter than PT.
- On Amazon (n=200), RB-NPMLE attains 97.6% coverage at width 0.226, about 49% shorter than Classical and 19% shorter than PT.
- In family-based GWAS, RB-NPMLE yields 731 overlaps with external signals versus 234 for direct-effect and 658 for population-effect baselines.
Surprising findings
- The naive Pred Mean intervals cover the truth only 37% of the time at alpha=0.10 on LM Arena, yet rebiasing still yields calibrated shorter intervals.
- RB-NPMLE makes slightly fewer total GWAS discoveries than the population-effect baseline but has more overlaps with external signals.
Limitations
- Assumes exchangeability of biases and that bias distribution does not depend on theta_i.
- Treats estimators as exactly normal with known variances; ignores CLT approximation error and plug-in variance estimation error.
- Coverage rates slow as the correlation gamma approaches 1; NPMLE can under-cover at stringent alpha in small-sample settings.
Experimental setup
Datasets: LM Arena human preference data (n=298 LLM pairs) Amazon product reviews (n=200 products) family-based GWAS summary statistics for human height (n=572,912 SNPs)
Benchmarks: LM Arena win-rate estimation Amazon product rating estimation synthetic Amazon PPI simulation family-based GWAS height discoveries
Metrics: empirical coverage average interval width width ratio vs Classical miscoverage rate number of BH discoveries and overlaps with external signals
Baselines: Classical Pred Mean PT (power-tuned PPI) PPI BH on direct-effect p-values BH on population-effect p-values
Builds on: prediction-powered inference power-tuned PPI NPMLE empirical Bayes Wald interval
Closest papers by meaning
- Mitigating Spurious Correlations via Disagreement Probability (NeurIPS 2024 · similarity 0.85)
- Kernel Debiased Plug-in Estimation: Simultaneous, Automated Debiasing without Influence Functions for Many Target Parameters (ICML 2024 · similarity 0.83)
- Statistical Inference Under Constrained Selection Bias (ICML 2024 · similarity 0.83)
- Identifying Structural Biases from Causal Mechanism Shifts (NeurIPS 2026 · similarity 0.83)
- Reducing Balancing Error for Causal Inference via Optimal Transport (ICML 2024 · similarity 0.83)
- Calibration without labels in multiple testing (NeurIPS 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.