Research map / Large language models / Language, safety and interpretability
High Accuracy, Less Talk (HALT): Reliable LLMs through Capability-Aligned Finetuning
Tim Franzmeyer, Archie Sravankumar, Lijuan Liu, Yuning Mao, Rui Hou, Sinong Wang, Jakob Nicolaus Foerster, Luke Zettlemoyer, Madian Khabsa
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| finetune on HALT capability-aligned data instead of unchanged data (n_capable) | Llama3-70B · Wikibios | F1: 0.35 -> 0.68 | Improved |
| finetune on HALT data instead of unchanged data (n_capable) | Llama3-70B · MATH | F1: 0.75 -> 0.87 | Improved |
| finetune on HALT data instead of unchanged data (n_capable) | Llama3-8B · Wikibios | F1: 0.26 -> 0.63 | Improved |
| finetune on HALT data vs unchanged (n_all) | Llama3-70B · APPS intro | F1: 0.44 -> 0.41 | Hurt |
| finetune on HALT data vs unchanged (n_all) | Mistral-7B · Wikibios | F1: 0.14 -> 0.13 | Hurt |
| replace removal by capability-aligned removal vs RandomTrim of same length (ablation) | Llama3-70B · Wikibios | F1 (n_all): 0.12 -> 0.33 | Improved |
| use HALT instead of IDK abstention (ablation) | Llama3-70B · APPS intro | F1 (n_all): 0.19 -> 0.41 | Improved |
| train combined multi-domain model with HALT alpha=40% | Llama3-70B · four domains | correctness (%): 51 -> 87 | Improved |
| train combined multi-domain model with HALT alpha=40% | Llama3-70B · four domains | completeness (%): 51 -> 25 | Hurt |
| lower alpha percentile of selected preliminary response (ablation) | HALT Llama3-70B · four datasets | correctness: +-17% | Improved |
| HALT-finetune vs base model | Llama3-70B · AlpacaEval 578 prompts | win rate: 50.8% | No clear effect |
| HALT-finetune vs base model | Llama3-70B · Wikibios AlpacaEval | win rate: 66.0% | Improved |
Closest papers by meaning
- Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs (ICLR 2026 · similarity 0.90)
- HonestLLM: Toward an Honest and Helpful Large Language Model (NeurIPS 2024 · similarity 0.90)
- Your Pre-trained LLM is Secretly an Unsupervised Confidence Calibrator (NeurIPS 2025 · similarity 0.90)
- Varying Shades of Wrong: Aligning LLMs with Wrong Answers Only (ICLR 2025 · similarity 0.90)
- Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMs (NeurIPS 2025 · similarity 0.89)
- Fixing It in Post: A Comparative Study of LLM Post-Training Data Quality and Model Performance (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
