Research map / Probabilistic and scientific modeling / Molecules and proteins
Conditional Testing based on Localized Conformal $p$-values
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Defines localized conformal p-values by inverting randomly localized conformal prediction intervals and applies them to conditional outlier detection (FDR), conditional label screening (FWER), and two-sample conditional distribution testing via a weighted U-statistic.
Problem
Conditional testing problems (outlier detection, label screening, two-sample conditional distribution testing) where standard conformal p-values target the marginal rather than the conditional distribution, so they lack power and conditional error control.
Approach
Localized conformal p-values are built by inverting randomly localized conformal prediction intervals, using kernel-weighted empirical distributions over calibration scores, so the p-value adapts to the conditional distribution of the score given covariates while keeping finite-sample validity via a random covariate draw.
Key results
- LCP-od controls FDR near nominal level in simulations and has higher power than CP in Scenario B1, where CP power stays flat while LCP approaches 1 as n grows.
- On health indicator screening, LCP-ls achieves conditional FWER 0.0329 (LL) versus 0.0740 for THR at nominal alpha=0.05, while both keep marginal FWER near 0.05.
- LCT controls type I error near 0.05 across all scenarios and classifiers, whereas CT inflates type I error under misspecified NN density ratio models (e.g., 0.441 on airfoil case (i)).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use localized conformal p-values instead of unweighted conformal p-values | Conditional outlier detection with CQR-QRF · House Sales, quantile-based outliers, α=0.15 | Power: 0.858 (0.070) -> 0.871 (0.048) | Helped |
| Use localized conformal p-values instead of unweighted conformal p-values | Conditional outlier detection with CQR-QRF · House Sales, quantile-based outliers, α=0.2 | Power: 0.907 (0.022) -> 0.913 (0.021) | Helped |
| Use localized conformal p-values instead of unweighted conformal p-values | Conditional outlier detection with CQR-QRF · House Sales, variance-based outliers, α=0.15 | Power: 0.581 (0.119) -> 0.628 (0.092) | Helped |
| Use localized conformal p-values instead of unweighted conformal p-values | Conditional outlier detection with CQR-QRF · House Sales, variance-based outliers, α=0.2 | Power: 0.692 (0.106) -> 0.722 (0.056) | Helped |
| Use localized p-values instead of unweighted CP thresholding | Conditional label screening with LL · Health indicators | Conditional FWER: 0.0740 (0.028) -> 0.0329 (0.018) | Helped |
| Use localized p-values instead of unweighted CP thresholding | Conditional label screening with RF · Health indicators | Conditional FWER: 0.0887 (0.029) -> 0.0372 (0.018) | Helped |
| Use localized conformal testing instead of CT | Two-sample conditional distribution test · Airfoil, Case (i), NN | Rejection proportion: 0.441 (0.0610) -> 0.053 (0.0374) | Helped |
| Use localized conformal testing instead of CT | Two-sample conditional distribution test · Airfoil, Case (ii), NN | Rejection proportion: 0.135 (0.0405) -> 0.066 (0.0390) | Helped |
| Use localized conformal testing instead of DCT | Two-sample conditional distribution test · Airfoil, Case (iii), NN | Median p-value: 0.018 -> 0.000 | Helped |
| Use localized conformal testing instead of DCT | Two-sample conditional distribution test · Airfoil, Case (iii), LL | Median p-value: 0.000 -> 0.000 | No clear effect |
| Use localized conformal p-values instead of unweighted CP p-values | Conditional outlier detection with CQR-QNN · House Sales, quantile-based outliers, α=0.15 | Power: 0.593 (0.037) -> 0.657 (0.096) | Helped |
Surprising findings
- The conformal test CT has type I error of 0.441 with an NN density ratio estimator on airfoil case (i), showing severe inflation under misspecification.
Limitations
- Random sampling of covariates introduces external randomness that reduces stability of the methods.
- Localization reduces effective sample size, so power is low when the sample size is small.
- Uses covariates of sampled point rather than the test point itself, which differs in finite samples.
Experimental setup
Datasets: House Sales in King County Health indicators (Kaggle) Airfoil self-noise (UCI)
Benchmarks: House Sales synthetic outliers Health indicators label screening Airfoil two-sample test synthetic outlier detection, label screening, and two-sample scenarios
Metrics: FDR power FWER (marginal and conditional) type I error (size) rejection proportion median p-value
Baselines: CP (unweighted conformal p-value with BH) THR (unweighted thresholding) CT (conformal test) DCT (de-biased conformal test)
Builds on: conformal prediction randomly localized conformal prediction weighted conformal p-values Benjamini-Hochberg procedure conditional calibration
Closest papers by meaning
- Conformal Prediction with Cellwise Outliers: A Detect-then-Impute Approach (ICML 2025 · similarity 0.88)
- Conformalized Multiple Testing after Data-dependent Selection (NeurIPS 2024 · similarity 0.88)
- Probabilistic Conformal Prediction with Approximate Conditional Validity (ICLR 2025 · similarity 0.87)
- Rectifying Conformity Scores for Better Conditional Coverage (ICML 2025 · similarity 0.87)
- Calibration without labels in multiple testing (NeurIPS 2026 · similarity 0.87)
- Robust Conditional Conformal Prediction via Branched Normalizing Flow (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.