Research map / Label-efficient and robust learning / Domain adaptation and generalization

Conformal Inference under High-Dimensional Covariate Shifts via Likelihood-Ratio Regularization

NeurIPS 2025 · Label-efficient and robust learning › Domain adaptation and generalization · approach: Mixture and distillation for domain shift · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes likelihood-ratio regularized quantile regression (LR-QR), a conformal method for covariate shift that avoids estimating the likelihood ratio by adding a data-dependent regularizer to the pinball loss, with finite-sample coverage bounds via stability analysis.

Problem

Conformal prediction under covariate shift in high-dimensional settings, where estimating the likelihood ratio is unreliable and coverage in the target domain can fail.

Approach

Solve a joint quantile regression over a linear class and a scalar beta with regularizer lambda(E1[beta^2 h^2] - E2[2 beta h]), whose stationarity conditions give target-domain coverage for the projection of the likelihood ratio onto the class, so the likelihood ratio itself never needs to be estimated.

Key results

• On MMLU with Llama 13B, LR-QR reaches 89.6% coverage (nominal 90%) with set size 3.38, versus WCP 86.5% (3.31) and SCP 78.1% (2.60); DRO over-covers at 99.7% with larger sets.
• On the Communities and Crime regression task, LR-QR is reported to stay closer to nominal coverage than weighted CP, 2R-CP and DRO across four racial-subgroup shifts.
• On RxRx1, LR-QR tracks the 0.9 nominal coverage more closely than the baselines across 14 target experiments.

Measured effects reported in the paper

Surprising findings

• Split conformal undercovers in some setups and overcovers in others on Communities and Crime, and DRO over-covers strongly on MMLU (99.7%).

Limitations

• Coverage guarantee is marginal and holds only up to an error term from restricting the hypothesis class; the projection error E1[(r - r_B)^2] is not eliminated.
• Regularization strength must be tuned (cross-validation on gradient norm), and the theoretically optimal lambda depends on sample sizes.
• Theory requires several regularity conditions (bounded features, strong convexity with high probability) that may not hold in practice.

Experimental setup

Datasets: Communities and Crime RxRx1 (WILDS) MMLU

Benchmarks: Communities and Crime RxRx1 (WILDS) MMLU

Metrics: marginal coverage average prediction set size

Baselines: split conformal prediction weighted conformal prediction 2R-CP DRO-CP DR-iso Robust-CP

Builds on: weighted conformal prediction pinball loss quantile regression stability bounds

Closest papers by meaning

• Co-optimization for Adaptive Conformal Prediction (NeurIPS 2026 · similarity 0.88)
• Rectifying Conformity Scores for Better Conditional Coverage (ICML 2025 · similarity 0.87)
• Kandinsky Conformal Prediction: Beyond Class- and Covariate-Conditional Coverage (ICML 2025 · similarity 0.87)
• Conformal Prediction under Lévy-Prokhorov Distribution Shifts: Robustness to Local and Global Perturbations (NeurIPS 2025 · similarity 0.87)
• A Kernel Nonconformity Score for Multivariate Conformal Prediction (NeurIPS 2026 · similarity 0.87)
• Wasserstein-Regularized Conformal Prediction under General Distribution Shift (ICLR 2025 · similarity 0.86)