Research map / Large language models / Language, safety and interpretability
NoVo: Norm Voting off Hallucinations with Attention Heads in Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NoVo, an inference-only method that selects attention-head L2 norms correlated with truth using 30 random samples and combines them by majority vote for zero-shot multiple-choice QA.
Problem
LLM hallucinations in zero-shot multiple-choice QA; existing representation editing and reading methods need specialised tools, in-domain training, or curated resources, which limits scalability and generalization.
Approach
Uses the L2 norm of each attention head's final-position context vector as a continuous truth proxy; selects truth-correlated heads (with correlation direction) using accuracy on only 30 random training samples, then performs a majority vote over selected heads, bypassing the likelihood layer.
Key results
- NoVo achieves 78.09% on TruthfulQA MC1 with Mistral-7B-Instruct, surpassing the prior best REAR method by at least 19 points and GPT-4 by 19 points.
- NoVo improves over the LM baseline on most of 20 datasets, with significant gains on over 90% of them (e.g., QASC 66.09 vs 31.53 LM for Mistral-7B-Instruct).
- On DeBERTa-Large finetuning, +NoVo outperforms SFT on all datasets except HSwag, by an average of 1.3 points.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply NoVo norm-voting to Mistral-7B-Instruct | Mistral-7B-Instruct · TruthfulQA MC1 | accuracy: 53.86 -> 78.09 | Helped |
| apply NoVo norm-voting | Mistral-7B-Instruct · QASC | accuracy: 31.53 -> 66.09 | Helped |
| apply NoVo norm-voting | Mistral-7B-Instruct · HellaSwag | accuracy: 75.28 -> 63.35 | Hurt |
| apply NoVo norm-voting | Mistral-7B-Instruct · PIQA | accuracy: 76.39 -> 76.66 | No clear effect |
| replace percentile-threshold majority voting with hyperparameter-free weighted voting (ablation) | NoVo on Mistral-7B-Instruct · TruthfulQA MC1 | accuracy: 78.09 -> 79.44 | Helped |
| use per-voter norm correlation direction rather than a fixed highest-norm direction (ablation) | NoVo on Mistral-7B-Instruct · TruthfulQA MC1 | accuracy: 72.58 -> 78.09 | Helped |
| use per-voter norm correlation direction rather than a fixed lowest-norm direction (ablation) | NoVo on Mistral-7B-Instruct · TruthfulQA MC1 | accuracy: 76.74 -> 78.09 | Helped |
| remove the period at sequence end (ablation) | NoVo · TruthfulQA MC1 | accuracy: -25.34 | Hurt |
| remove the period at sequence end (ablation) | NoVo · QASC | accuracy: -34.87 | Hurt |
| remove the period at sequence end (ablation) | NoVo · HellaSwag | accuracy: -16.21 | Hurt |
| remove the period at sequence end (ablation) | NoVo · SIQA | accuracy: 0 | No clear effect |
| remove the period at sequence end (ablation) | NoVo · PIQA | accuracy: 0 | No clear effect |
Surprising findings
- NoVo accuracy drops on HellaSwag and PIQA, where questions require broad real-world assumptions.
- Using a single sample set of 30 random samples gives high variance; the 200-run analysis shows std below about 1.5 points for most datasets.
Limitations
- Accuracy drops on HellaSwag and PIQA, where broad assumptions are needed.
- Performance is bounded by underlying model capability (e.g., confusion-people category remains inaccurate).
- Naive ensemble plateaus after about 240 voters; results depend on random 30-sample selection and hyperparameters (30 samples, 85th percentile).
Experimental setup
Datasets: TruthfulQA MC1 Arc-Easy CommonsenseQA 2.0 QASC SWAG HellaSwag SIQA PIQA CosmosQA CICERO v1 CICERO v2 Adversarial GLUE
Benchmarks: TruthfulQA MC1 CommonsenseQA 2.0 QASC SWAG HellaSwag SIQA PIQA CosmosQA CICERO v1 CICERO v2 Adversarial GLUE FACTOR-Expert
Metrics: accuracy
Baselines: LM log likelihood TruthX ITI TrFr DoLa ICD RePE GPT-4 TEAM SFT
Builds on: Inference Time Intervention DoLa RePE TEAM
Closest papers by meaning
- FactTest: Factuality Testing in Large Language Models with Finite-Sample and Distribution-Free Guarantees (ICML 2025 · similarity 0.89)
- LLM-Check: Investigating Detection of Hallucinations in Large Language Models (NeurIPS 2024 · similarity 0.89)
- In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation (ICML 2024 · similarity 0.88)
- THRONE: An Object-based Hallucination Benchmark for the Free-form Generations of Large Vision-Language Models (CVPR 2024 · similarity 0.88)
- ANAH-v2: Scaling Analytical Hallucination Annotation of Large Language Models (NeurIPS 2024 · similarity 0.88)
- Detecting and Preventing Hallucinations in Large Vision Language Models (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.