Research map / Large language models / Language, safety and interpretability
Conformal Language Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes conformal language modeling, calibrating a sampling stopping rule and rejection rule via Learn Then Test to produce LM output sets with distribution-free coverage guarantees, plus conformal selection of individually correct components.
Problem
Generative LMs have unbounded combinatorial output spaces, so standard conformal prediction cannot enumerate candidates; the paper seeks prediction sets for free-text generations with statistical coverage guarantees.
Approach
Calibrate hyperparameters of a sampling algorithm (diversity threshold, likelihood threshold, set-confidence stopping threshold) rather than enumerating outputs, using Learn Then Test with binomial tail p-values and Pareto testing; additionally calibrate a component-level selection threshold to output independently correct sentences.
Key results
- Conformal sampling with rejection empirically keeps average set loss at or below the target risk level across tasks, matching the theory (validity, though not conservative).
- On TriviaQA with LLaMA-13B, the AUC of expected set size for Max and Sum scoring is less than half that of the First-K baseline.
- Across tasks, Max scoring largely outperforms Sum and First-K on relative excess samples; component-level span-logits and classifier scores yield larger confident component sets than a random baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use likelihood-based Max or Sum scoring instead of uniform First-K scoring | Conformal sampling · TriviaQA, CNN/DM, MIMIC-CXR | expected set size | Helped |
| Use Max or Sum scoring instead of First-K scoring | Conformal sampling · TriviaQA | AUC of expected set size: less than half | Helped |
| Use First-K scoring instead of likelihood-based scoring | Conformal sampling · Long-output tasks | output set size | No clear effect |
| Use Max scoring instead of Sum or First-K scoring | Conformal sampling · Long-output tasks | relative number of excess samples | Helped |
| Add duplicate rejection to First-K scoring (ablation) | Conformal sampling · Long-output tasks | output set size | No clear effect |
| Add duplicate rejection to First-K scoring (ablation) | Conformal sampling · Long-output tasks | sampling efficiency | Hurt |
| Use likelihood-based Max, Sum, or First-K-reject scoring rather than First-K | Conformal sampling · All evaluated tasks | set loss relative to target risk | Helped |
| Model components independently rather than using the Random component-score baseline | Conformal component selection · Text summarization and radiology report generation | number of confident components | Helped |
Limitations
- Guarantees depend on i.i.d. prompts and on the admission function being a good proxy for quality; automatic admission functions (CheXbert, ROUGE) are imperfect.
- With finite k_max, low epsilon levels may be unachievable and the method returns null.
- Guarantees are marginal probabilistic statements and do not necessarily hold conditioned on specific input types.
Experimental setup
Datasets: MIMIC-CXR MIMIC-CXR-JPG CNN/DM TriviaQA MNLI SNLI FEVER SciTail PAWS VitaminC
Benchmarks: MIMIC-CXR CNN/DM TriviaQA
Metrics: set loss vs target epsilon normalized AUC of excess sample fraction normalized AUC of prediction set size component recall
Baselines: First-K First-K+reject Max scoring Sum scoring Random component baseline
Builds on: Learn Then Test split conformal prediction Pareto testing fixed sequence testing
Closest papers by meaning
- Large language model validity via enhanced conformal prediction methods (NeurIPS 2024 · similarity 0.93)
- Language Models with Conformal Factuality Guarantees (ICML 2024 · similarity 0.90)
- Geometry-Calibrated Conformal Abstention for Language Models (NeurIPS 2026 · similarity 0.89)
- Conformal Language Modeling via Posterior Sampling (NeurIPS 2026 · similarity 0.88)
- Sample then Identify: A General Framework for Risk Control and Assessment in Multimodal Large Language Models (ICLR 2025 · similarity 0.87)
- Prune 'n Predict: Optimizing LLM Decision-making with Conformal Prediction (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.