Research map / Large language models / Reasoning and chain of thought
Calibrating Reasoning in Language Models with Internal Consistency
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes internal consistency, the agreement of logit-lens latent predictions across layers, as a training-free confidence measure to reweight sampled CoT reasoning paths (SC+IC), improving calibrated accuracy on four true/false reasoning datasets.
Problem
CoT reasoning in LLMs often yields unfaithful or contradictory rationales, and existing calibration of reasoning paths relies on final-layer logits; the paper asks whether internal representations across layers can measure reliability of reasoning paths.
Approach
Measure confidence as the fraction of intermediate layers whose logit-lens latent predictions at the answer token agree with the final prediction, and use this score to weight sampled reasoning paths in self-consistency voting, with optional learned per-layer weights.
Key results
- SC+IC improves mean calibrated accuracy over SC with CoT prompting by about 0.6 to 1.5 points across the four models (e.g., Llama-2-7B 61.8 to 62.4; Mistral-7B 74.6 to 76.1), with reported improvements of 1.8% to 4.9% in the text.
- Tuned layer weights (SC+IC tune) raise Mixtral-8x7B mean CoT calibrated accuracy from 79.6 (SC) to 81.1, and transfer weights from PrOntoQA match tuned results closely.
- With least-to-most prompting, SC+IC (tune) lifts Llama-2-7B mean accuracy from 69.8 (SC) to 73.4.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| up-weight sampled CoT reasoning paths by internal consistency (SC+IC) instead of plain majority vote (SC) | Llama-2-7B, Llama-2-13B, Mistral-7B, Mixtral-8x7B · CoT, 4 datasets mean | calibrated accuracy: +0.6 to +1.5 mean points over SC (Llama-2-7B 61.8→62.4; 13B 68.9→70.6; Mistral 74.6→76.1; Mixtral 79.6→80.6) | Helped |
| up-weight sampled reasoning paths by internal consistency (SC+IC) instead of SC | Llama-2-7B on PrOntoQA CoT · PrOntoQA CoT | calibrated accuracy: 49.0 vs 49.0 | No clear effect |
| learned layer-weight vector for internal consistency (SC+IC tune) instead of uniform layer weights (ablation) | Llama-2-7B, L2M prompting · L2M, 4 datasets mean | calibrated accuracy: 71.0 → 73.4 | Helped |
| layer weights tuned on PrOntoQA transferred to other datasets (SC+IC transfer) vs tuned per dataset (ablation) | Mixtral-8x7B, CoT · CoT, 4 datasets mean | calibrated accuracy: 81.1 (transfer) vs 81.1 (tune) | No clear effect |
| learned layer weights (SC+IC tune) on top of uniform internal consistency (ablation) | Mistral-7B, CoT · CoT, 4 datasets mean | calibrated accuracy: 76.1 → 76.8 | Helped |
| chain-of-thought prompting vs few-shot without CoT: CoT decreases internal consistency | LLMs on PrOntoQA · PrOntoQA probe/consistency analysis | internal consistency | Hurt |
| SC+IC (layer-weighted, transfer) vs SC+Δ logit-based baseline | Llama-2-13B, CoT · CoT, 4 datasets mean | calibrated accuracy: 70.6 vs 69.9 (SC+Δ) | Helped |
| up-weight paths by internal consistency vs logit-based SC+Δ on Mixtral CoT | Mixtral-8x7B, CoT · CoT, BoolQ | calibrated accuracy: 81.7 vs 82.6 (SC+Δ) | Hurt |
Surprising findings
- CoT prompting decreases internal consistency relative to direct prompting, even though it improves answer accuracy.
- Decoded latent predictions from intermediate layers are heavily skewed toward 'True', requiring per-layer median balancing before use.
Limitations
- Restricted to decoder-only models; encoder-decoder extension (decoder lens) left for future work.
- Only vanilla CoT and least-to-most prompting studied; other prompting techniques untested.
- Tasks restricted to true/false questions with single-token answers.
Experimental setup
Datasets: BoolQ CoinFlip PrOntoQA ProofWriter
Benchmarks: BoolQ CoinFlip PrOntoQA ProofWriter
Metrics: calibrated accuracy agreement ratio calibration plot
Baselines: Greedy SC SC+Δ least-to-most prompting
Builds on: self-consistency logit lens least-to-most prompting chain-of-thought prompting
Closest papers by meaning
- Improving Factuality and Reasoning in Language Models through Multiagent Debate (ICML 2024 · similarity 0.91)
- Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes (ICLR 2026 · similarity 0.91)
- The Tell-Tale Norm: $\ell_2$ Magnitude as a Signal for Reasoning Dynamics in Large Language Models (ICML 2026 · similarity 0.90)
- Potemkin Understanding in Large Language Models (ICML 2025 · similarity 0.90)
- Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning (ICML 2025 · similarity 0.90)
- Improving Reasoning Performance in Large Language Models via Representation Engineering (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.