Research map / Probabilistic and scientific modeling / Molecules and proteins
Active Statistical Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
placeholder
Problem
Statistical inference (confidence intervals and hypothesis tests) about label-dependent parameters when labels are costly and only a labeling budget is available; uniform labeling wastes budget.
Approach
Sample labels with probability proportional to a model-derived uncertainty estimate (predicted |f(X)-Y|), and form an AIPW-style debiased M-estimator with inverse-propensity weights, which stays valid for any black-box model while shrinking variance; extended to sequential settings with martingale CLTs and online fine-tuning.
Key results
- On Pew survey, active inference saves over 85% of budget vs classical inference and around 25% vs uniform PPI for Biden approval.
- On census regression, active inference saves around 80% of budget over classical inference and over 60% over the uniform baseline.
- On AlphaFold proteomics odds ratio, over 75% budget saved vs classical and around 20-25% vs uniform PPI, with valid coverage.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use model-uncertainty-guided label collection instead of uniform random sampling | Batch active inference · Pew 2020 election survey — Biden and Trump approval | confidence interval width: at least $\sim 10\%$ | Helped |
| Use model-uncertainty-guided label collection instead of uniform random sampling | Batch active inference · California ACS PUMS — age coefficient for income | confidence interval width | Helped |
| Use model-uncertainty-guided label collection instead of uniform random sampling | Batch active inference · Proteomics — phosphorylation and IDR odds ratio | confidence interval width | Helped |
| Fine-tune the predictive model during label collection (ablation) | Sequential active inference · Pew 2020 election survey — Biden and Trump approval | budget required for matched confidence interval width: Biden: around $40\%$ vs. no fine-tuning and $30\%$ vs. PPI; Trump: around $45\%$ and $35\% | Helped |
| Fine-tune the prediction and error models during label collection (ablation) | Sequential active inference · California ACS PUMS — age coefficient for income | budget required for matched confidence interval width: around $32-40\%$ vs. no fine-tuning and around $20-30\%$ vs. the uniform baseline | Helped |
| Use active sampling with a poorly trained model instead of uniform sampling (ablation) | Sequential active inference without fine-tuning · Pew 2020 election survey — approval | inferential power | Hurt |
| Use non-asymptotic confidence intervals instead of asymptotic intervals (ablation) | Batch active inference · Pew 2020 election survey — approval | confidence interval width | Mixed |
Surprising findings
- Active inference with no fine-tuning can be worse than uniform PPI when the predictive model is poorly trained (10 labels), and fine-tuning remedies this.
Limitations
- Asymptotic validity relies on convergence of the sampling rule and model; non-asymptotic results only for mean estimation with bounded observations.
- Requires consistency of the M-estimator and smoothness assumptions; the sequential variant depends on model convergence.
- Uniform mixing parameter tau must be tuned or set heuristically.
Experimental setup
Datasets: Pew Research Center post-2020 election survey ACS PUMS California 2019 (via Folktables) AlphaFold post-processed IDR predictions for proteomics
Benchmarks: Pew 2020 election approval survey ACS PUMS census income regression AlphaFold phosphorylation-IDR odds ratio
Metrics: confidence interval width empirical coverage percentage of sample budget saved
Baselines: prediction-powered inference (uniform sampling) classical inference (no ML)
Builds on: prediction-powered inference augmented inverse propensity weighting M-estimation asymptotic normality martingale central limit theorem
Closest papers by meaning
- Robust Sampling for Active Statistical Inference (NeurIPS 2025 · similarity 0.91)
- Post-ADC Inference: Valid Inference After Active Data Collection (NeurIPS 2026 · similarity 0.88)
- Uncertainty Herding: One Active Learning Method for All Label Budgets (ICLR 2025 · similarity 0.85)
- Scaling Up Active Testing to Large Language Models (NeurIPS 2025 · similarity 0.85)
- Prediction-Powered Active Testing (NeurIPS 2026 · similarity 0.85)
- Task-Agnostic Machine-Learning-Assisted Inference (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.