Research map / Probabilistic and scientific modeling / Molecules and proteins
Enhancing Statistical Validity and Power in Hybrid Controlled Trials: A Randomization Inference Approach with Conformal Selective Borrowing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Fisher randomization tests with a conformal selective borrowing estimator (CSB) that individually tests and pools external controls for hybrid RCT analysis, with an MSE-minimizing adaptive selection threshold.
Problem
Hybrid controlled trials that borrow non-randomized external controls (ECs) can suffer hidden bias and inflated type I error, especially with small randomized samples, while asymptotic and Bayesian methods lack finite-sample validity.
Approach
Use Fisher randomization tests (only RCT randomization as the reference distribution) with a test statistic built from a doubly robust ATE estimator that selectively borrows ECs whose conformal p-values for exchangeability with RCT controls exceed a threshold, and choose that threshold by bootstrap-estimated MSE.
Key results
- In the CALGB 9633 + NCDB analysis, CSB (264 ECs borrowed) gave estimate 0.138 with exact p=0.046, while full borrowing gave 0.241 with exact p=0.031 and no-borrowing AIPW gave exact p=0.051.
- In simulations with no hidden bias (b=0), FB and CSB reduced MSE by 42% and 20% and increased power by 46% and 45% relative to NB, with type I error controlled.
- With hidden bias b=1-8, FB showed bias of 125%-203% of its SD and up to 454% MSE increase over NB, while CSB decreased MSE by 13%-16% for most bias levels (except b=1,2).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Full Borrow instead of No Borrow | FRT for hybrid controlled trials · Simulations with no hidden bias (b=0) | power: +46% | Helped |
| Use Conformal Selective Borrowing instead of No Borrow | FRT for hybrid controlled trials · Simulations with no hidden bias (b=0) | power: +45% | Helped |
| Use Full Borrow instead of No Borrow | ATE estimation · Simulations with no hidden bias (b=0) | MSE: 42% | Helped |
| Use Conformal Selective Borrowing instead of No Borrow | ATE estimation · Simulations with no hidden bias (b=0) | MSE: 20% | Helped |
| Use Full Borrow instead of No Borrow | FRT for hybrid controlled trials · Simulations with hidden bias (b = 1, ..., 8) | power: up to 51% | Hurt |
| Use Conformal Selective Borrowing instead of No Borrow | FRT for hybrid controlled trials · Simulations with hidden bias (b = 1, ..., 8) | power: increases by 13%-36%; decreases by 7%-20% | Mixed |
| Use Conformal Selective Borrowing instead of No Borrow | ATE estimation · Simulations with hidden bias (b = 1, ..., 8) | MSE: decreases by 13%-16%; increases by 1%-18% | Mixed |
| Use Full Borrow instead of No Borrow | ATE estimation · Simulations with hidden bias (b = 1, ..., 8) | MSE: up to 454% | Hurt |
| Use the adaptive CSB selection threshold instead of fixed threshold γ = 0.6 (ablation) | CSB + FRT · Simulations with varying hidden bias | power | Mixed |
| Borrow all external controls with Full Borrow rather than selectively borrowing with CSB | FRT for hybrid controlled trials · Simulations with no hidden bias | power | Helped |
| Borrow all external controls with Full Borrow rather than using No Borrow | FRT for hybrid controlled trials · Simulations with unmeasured confounding | power | Hurt |
Surprising findings
- Full borrowing with asymptotic inference gives p<0.001 on the real data, but its exact randomization p-value is 0.031, showing that asymptotic p-values can overstate significance.
- The adaptive CSB estimator can lose power relative to no borrowing when bias is non-negligible but hard to detect (b=2,3,4), a cost of transferability detection.
Limitations
- When hidden bias is non-negligible yet difficult to detect, CSB may lose power relative to no borrowing.
- Conformal p-values require exchangeability (stronger than mean exchangeability) between ECs and randomized controls; the factor of 2 in the CV+/Jackknife+ bound cannot be reduced without further assumptions.
- Real-data analysis relies on pseudo-observations for RMST and covariate-dependent censoring adjustment, and results are only validated on one trial.
Experimental setup
Datasets: CALGB 9633 National Cancer Database (NCDB)
Benchmarks: simulated hybrid trials with hidden bias CALGB 9633 + NCDB lung cancer trial
Metrics: type I error rate power MSE bias exact p-value asymptotic p-value
Baselines: No Borrowing (NB) Full Borrowing (FB) Adaptive Lasso Selective Borrowing (ALSB) Difference-in-means
Builds on: Fisher randomization test conformal inference CV+ / Jackknife+ doubly robust estimator for hybrid controlled trials adaptive lasso selective borrowing
Closest papers by meaning
- Adaptive Data-Borrowing for Improving Treatment Effect Estimation using External Controls (NeurIPS 2025 · similarity 0.89)
- Data-driven Design of Randomized Control Trials with Guaranteed Treatment Effects (ICML 2025 · similarity 0.85)
- Doubly Protected Estimation for Survival Outcomes Utilizing External Controls for Randomized Clinical Trials (ICML 2025 · similarity 0.84)
- When Selection Meets Intervention: Additional Complexities in Causal Discovery (ICLR 2025 · similarity 0.84)
- Conformalized Multiple Testing after Data-dependent Selection (NeurIPS 2024 · similarity 0.84)
- Controllable Generative Sandbox for Causal Inference (NeurIPS 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.