Research map / Large language models / Language, safety and interpretability
Conformal Language Model Reasoning with Coherent Factuality
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Defines coherent factuality for reasoning outputs and proposes a conformal prediction method that filters LM reasoning claims over GPT-4o-generated deducibility graphs, guaranteeing coherent factuality at user-specified rates.
Problem
Conformal factuality methods treat claims as independent, which fails for multi-step reasoning where each step depends on preceding steps; the paper needs a coherent notion of factuality and a calibrated filtering method for it.
Approach
Defines coherent factuality (each claim deducible from its prefix, ground truth, and question), represents inter-claim dependence as an approximate deducibility graph generated by GPT-4o, and applies split conformal prediction over ancestor-connected subgraphs with a descendant-weighted risk score.
Key results
- Subgraph filtering achieves 90% factuality while retaining close to 80% of claims on MATH.
- Both theoretical calibration bounds hold empirically across factuality levels when calibrated on silver annotations and validated on gold annotations.
- At alpha=0.1, 64% of error-free outputs contain a correct final answer, the same rate as the baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use coherently filtered claims instead of independently filtered claims as reprompt context | GPT output generation · Reprompting, α = 0.05 | error-rate reduction: ↓18% vs. ↓2% | Helped |
| Use coherently filtered claims instead of independently filtered claims as reprompt context | GPT output generation · Reprompting, α = 0.10 | error-rate reduction: ↓17.12% vs. ↓11.44% | Helped |
| Use coherently filtered claims instead of independently filtered claims as reprompt context | GPT output generation · Reprompting, α = 0.15 | error-rate reduction: ↓14% vs. ↓9.16% | Helped |
| Use deducibility-graph subgraph filtering rather than independent factuality filtering | Coherent claim filtering · MATH | coherent factuality calibration | Helped |
| Use post-hoc graph filtering rather than calibrated subgraph filtering | Coherent claim filtering · MATH | claim retention at a user-specified factuality rate | Hurt |
| Use coherent filtering rather than the baseline independent filtering | GPT-4 and Llama-3.1-70B-Instruct output grading · MATH | legibility (false positive and false negative rates) | Helped |
Surprising findings
- GPT-4o-generated deducibility graphs were found to have more empirical utility than ideal human-constructed graphs.
- Independent-factuality baseline fails to achieve calibration and competitive claim retention on reasoning outputs.
Limitations
- Reliance on proprietary GPT-4o for accurate graph generation; GPT-4o struggled with long outputs with many claims.
- Ground truth and deducibility are subjective and context-dependent.
- Llama-generated graphs satisfied the approximate deducibility definition for only 40% of examples in the checked subset.
Experimental setup
Datasets: MATH FELM
Benchmarks: MATH FELM
Metrics: factuality rate claim retention calibration error error reduction on reprompting
Baselines: conformal factuality (Mohri and Hashimoto) linear deducibility graph post-hoc filtering no dependency baseline
Builds on: conformal factuality (Mohri and Hashimoto) split conformal prediction self-consistency
Closest papers by meaning
- Inference-Time Conformal Reasoning with Valid Factuality Control for Large Language Models (ICML 2026 · similarity 0.92)
- Language Models with Conformal Factuality Guarantees (ICML 2024 · similarity 0.90)
- Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning (ICLR 2024 · similarity 0.87)
- Logical Consistency of Large Language Models in Fact-Checking (ICLR 2025 · similarity 0.87)
- Are Language Models Efficient Reasoners? A Perspective from Logic Programming (NeurIPS 2025 · similarity 0.87)
- Pause and Reflect: Conformal Aggregation for Chain-of-Thought Reasoning (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.