Research map / Large language models / Language, safety and interpretability
Multivariate Conformal Selection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends conformal selection to multivariate responses via regionally monotone nonconformity scores (distance-based mCS-dist and learned mCS-learn), with finite-sample FDR control via BH on conformal p-values.
Problem
Selecting test candidates whose multivariate responses lie in a target region while controlling false discovery rate; existing conformal selection only handles univariate responses with threshold criteria.
Approach
Introduces regional monotonicity for nonconformity scores so conformal p-values are conservative for null candidates, enabling BH-based FDR control for arbitrary closed target regions; proposes a clipped distance-based score and a learned score trained with a differentiable smooth-p-value loss.
Key results
- On simulated orthant task (q=0.3), mCS-dist achieves highest power among FDR-controlling methods in all six settings (e.g., 0.555 vs 0.325 for bi/mCS-l in Setting 1).
- On real ADMET data, mCS-dist and mCS-learn achieve best and second-best power across all tasks; e.g., Task 3 at q=0.3 mCS-learn power 0.431 vs mCS-dist 0.410.
- CS_int violates FDR badly (observed FDR ~0.72-0.81 at q=0.3) while CS_ib is valid but overly conservative with zero power in simulations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use mCS-dist instead of bi | Task 1 shifted first orthant selection · Task 1 shifted first orthant, Settings 1–6 | power: 0.324 vs 0.194 (Setting 4) | Helped |
| Use mCS-learn instead of bi | Task 1 shifted first orthant selection · Task 1 shifted first orthant, Settings 1–6 | power: 0.102 vs 0.059 (Setting 3) | Helped |
| Use mCS-dist instead of bi | Task 2 spherical-region selection · Task 2 sphere, Settings 1–6 | power: 0.333 vs 0.192 (Setting 4) | Helped |
| Use mCS-learn instead of bi | Task 2 spherical-region selection · Task 2 sphere, Settings 1–6 | power: 0.179 vs 0.115 (Setting 3) | Helped |
| Use mCS-learn rather than mCS-dist | Task 1 shifted first orthant selection · Task 1 shifted first orthant, Settings 1–6 | power: 0.325 vs 0.555 (Setting 1); 0.108 vs 0.104 (Setting 2) | Mixed |
| Use mCS-learn rather than mCS-dist | Task 2 spherical-region selection · Task 2 sphere, Settings 1–6 | power: 0.534 vs 0.760 (Setting 1); 0.421 vs 0.405 (Setting 2) | Mixed |
| Use CS_int at nominal FDR level q | Task 1 selection · Task 1 shifted first orthant | FDR: 0.773 | Hurt |
| Use CS_is with hold-out validation | Task 1 selection · Task 1 shifted first orthant, Setting 6 | FDR: 0.374 | Hurt |
| Replace multivariate selection with binary-indicator baseline bi | Real-data ADMET selection · Task 1, q = 0.3 | power: 0.000 | Hurt |
| Use the clipped distance score rather than the regular distance score (ablation) | mCS-dist · Multivariate selection | power | Helped |
| Use the smooth conformal p-value loss L2 rather than smooth selection-size loss L1 (ablation) | mCS-learn · Multivariate selection | power | Helped |
Surprising findings
- Naive per-dimension CS with intersection (CS_int) yields observed FDR around 0.8 at nominal 0.3, far above the nominal level.
- The baseline bi nearly selects no compounds on real ADMET data, attributed to a weak classifier (F1 0.31).
Limitations
- mCS-dist underperforms on nonconvex target regions, motivating the learned score.
- Relies on a fixed predictor and an i.i.d./exchangeability assumption; regression focus, classification discussed only in appendix.
- Simulations use only 100 test points per iteration and a modest number of settings.
Experimental setup
Datasets: simulated Gaussian and t-distributed data (p=10, d up to 30) imputed public ADMET dataset (22805 compounds, 15 assays)
Benchmarks: simulated selection tasks (shifted orthant, sphere) ADMET drug discovery selection tasks
Metrics: FDR power
Baselines: CS_int CS_ib CS_is bi
Builds on: Conformal Selection Benjamini-Hochberg procedure conformal prediction
Closest papers by meaning
- Conformalized Multiple Testing after Data-dependent Selection (NeurIPS 2024 · similarity 0.89)
- A Kernel Nonconformity Score for Multivariate Conformal Prediction (NeurIPS 2026 · similarity 0.87)
- A Unified Comparative Study with Generalized Conformity Scores for Multi-Output Conformal Regression (ICML 2025 · similarity 0.86)
- Optimal transport-based conformal prediction (ICML 2025 · similarity 0.86)
- Multi-model Ensemble Conformal Prediction in Dynamic Environments (NeurIPS 2024 · similarity 0.85)
- Length Optimization in Conformal Prediction (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.