Research map / Label-efficient and robust learning / Recognition and long-tail
Domain-Shift-Aware Conformal Prediction for Large Language Models
Zhexiao Lin, Yuanyuan Li, Neeraj Sarna, Yuanyuan Gao, Michael Berger
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace standard CP with DS-CP (density-ratio weighting in embedding space plus regularized test weight) | standard conformal prediction on 16 open-source LLMs · MMLU cross-subject calibration/test (272 ordered subject pairs), alpha=0.10 | empirical coverage | Improved |
| replace standard CP with DS-CP | standard conformal prediction on 16 open-source LLMs · MMLU cross-subject (272 subject pairs) | average prediction set size | Hurt |
| apply DS-CP when standard CP already achieves valid coverage | DS-CP · MMLU cross-subject (272 subject pairs) | empirical coverage | No clear effect |
| apply DS-CP when standard CP severely under-covers | DS-CP · MMLU cross-subject (272 subject pairs) | empirical coverage | Improved |
| set regularizer lambda to the maximum estimated weight (versus gamma in 0 to 5 times max) (ablation) | DS-CP · MMLU cross-subject | coverage versus set size trade-off | Improved |
Closest papers by meaning
- Large language model validity via enhanced conformal prediction methods (NeurIPS 2024 · similarity 0.90)
- Conformal Language Modeling (ICLR 2024 · similarity 0.89)
- Language Models with Conformal Factuality Guarantees (ICML 2024 · similarity 0.89)
- Conformal Prediction with Paraphrase-Aware Scoring for LLM Uncertainty Quantification (NeurIPS 2026 · similarity 0.89)
- Conformal Prediction Beyond the Seen: A Missing Mass Perspective for Uncertainty Quantification in Generative Models (NeurIPS 2025 · similarity 0.88)
- Conformal Language Modeling via Posterior Sampling (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
