Research map / Large language models / Language, safety and interpretability
Linguistic Calibration of Long-Form Generations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Defines linguistic calibration for long-form LM generations via downstream user forecasts and trains Llama 2 7B with SFT on summary-distilled confidence statements followed by RL using a log-loss reward from a surrogate reader.
Problem
Long-form LM generations contain many claims and confident hallucinations mislead users' decisions, yet existing calibration methods only handle classification or short outputs, not multi-claim long-form text with verbalized confidence.
Approach
Defines linguistic calibration via calibration of a user's forecasts formed from the generation, and optimizes a strictly proper log-loss scoring rule on those forecasts end-to-end. Training is a summary-distillation SFT step followed by decision-based RL with a neural surrogate reader.
Key results
- On TriviaQA with simulated reader, LC RL reaches ECE 0.108 at 64.74% accuracy, versus 0.367 ECE and 63.33% accuracy for Factuality RL.
- On person biography generation (held-out task), LC RL reduces ECE to 0.266 from 0.601 for Factuality RL while improving accuracy to 46.77% from 39.86%.
- With human readers on TriviaQA, LC RL gets ECE 0.116 versus 0.404 for Factuality RL, with comparable accuracy (60.12% vs 59.62%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use linguistic-calibration RL instead of factuality RL | Llama 2 7B · TriviaQA (simulated reader) | ECE: 0.367 -> 0.108 | Helped |
| Use linguistic-calibration RL instead of factuality RL | Llama 2 7B · TriviaQA (simulated reader) | Accuracy (%): 63.33 -> 64.74 | Helped |
| Use linguistic-calibration RL instead of factuality RL | Llama 2 7B · Jeopardy (simulated reader) | ECE: 0.359 -> 0.088 | Helped |
| Use linguistic-calibration RL instead of factuality RL | Llama 2 7B · SciQ (simulated reader) | ECE: 0.439 -> 0.213 | Helped |
| Use linguistic-calibration RL instead of factuality RL | Llama 2 7B · BioASQ Task B (simulated reader) | ECE: 0.620 -> 0.342 | Helped |
| Use linguistic-calibration RL instead of factuality RL | Llama 2 7B · TriviaQA (human readers) | ECE: 0.404 -> 0.116 | Helped |
| Use linguistic-calibration RL instead of factuality RL | Llama 2 7B · Person Biography Generation (simulated reader) | ECE: 0.601 -> 0.266 | Helped |
| Use linguistic-calibration RL instead of factuality RL | Llama 2 7B · Person Biography Generation (simulated reader) | Accuracy (%): 39.86 -> 46.77 | Helped |
| Add decision-based RL to LC SFT (ablation) | Llama 2 7B · TriviaQA (simulated reader) | ECE: 0.166 -> 0.108 | Helped |
| Add decision-based RL to LC SFT (ablation) | Llama 2 7B · TriviaQA (simulated reader) | Accuracy (%): 60.98 -> 64.74 | Helped |
| Add decision-based RL to LC SFT (ablation) | Llama 2 7B · TriviaQA (human readers) | ECE: 0.163 -> 0.116 | Helped |
| Add decision-based RL to LC SFT (ablation) | Llama 2 7B · Person Biography Generation | ECE: 0.301 -> 0.266 | Helped |
Surprising findings
- LC SFT achieved higher accuracy than Factuality RL on person biography generation, attributed to more less-precise but correct claims.
- Llama 2 7B-based LC RL attains ECE competitive with GPT-4 baselines despite much lower factuality.
Limitations
- Many emitted confidence statements are unambiguous (e.g., percentages), so ambiguous linguistic confidence interpretation is not well studied.
- Off-the-shelf QA datasets are only a proxy for real-world decision-making query distributions.
- White-box finetuning is required; cannot calibrate API-only LLMs.
Experimental setup
Datasets: TriviaQA FactScore unlabeled person split Jeopardy SciQ BioASQ Task B
Benchmarks: TriviaQA Jeopardy SciQ BioASQ Task B FactScore person biography
Metrics: accuracy expected calibration error reliability diagrams
Baselines: Llama 2 Chat 7B Claude Distill ICL 8-Shot Factuality SFT Factuality RL Summary ICL 8-Shot GPT-4 0-Shot GPT-4 JAFU 0-Shot GPT-4 Summary ICL 8-Shot Direct Summary Eval
Builds on: Self-Consistency PPO FactScore Decision calibration (Zhao et al.)
Closest papers by meaning
- Calibrating Large Language Models with Sample Consistency (AAAI 2025 · similarity 0.89)
- Conformal Linguistic Calibration: Trading-off between Factuality and Specificity (NeurIPS 2025 · similarity 0.88)
- QA-Calibration of Language Model Confidence Scores (ICLR 2025 · similarity 0.87)
- Calibrating LLMs with Semantic-level Reward (NeurIPS 2026 · similarity 0.87)
- Improving Uncertainty Estimation through Semantically Diverse Language Generation (ICLR 2025 · similarity 0.87)
- Precise Information Control in Long-Form Text Generation (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.