Research map / Large language models / Instruction and fine-tuning
Truth Forest: Toward Multi-Scale Truthfulness in Large Language Models through Intervention without Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Truth Forest (TrFr), an inference-time intervention that trains multiple orthogonal linear probes on attention-head activations, sampled at random answer prefixes (Random Peek), to shift LLMs toward truthful answers without fine-tuning.
Problem
LLMs generate hallucinations, and the gap between discerning truth and generating it (the Generating-Discerning gap) limits existing probe-based truthfulness interventions that use only the last token and a single truth direction.
Approach
Learn multiple mutually orthogonal probe directions per attention head with an orthogonality penalty, combine them with exponential-decay weights, and apply a constant shift during inference; Random Peek samples features from random truncation points of answers instead of only the final token.
Key results
- On TruthfulQA, Llama 2-7B True rate rises from 40.8% to 74.5% with TrFr (True*Info 37.5% to 56.0%).
- On LLaMA-7B full-data setting, TrFr Single Mass reaches 63.2% True*Info and 77.2% True, versus 42.1% and 45.4% for ITI mass mean shift.
- Ablation on LLaMA-7B: orthogonal directions alone raise True*Info from 29.6% to 36.7%; adding Random Peek raises it to 49.7%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply TrFr | LLaMA-7B · TruthfulQA, full data | True*Info (%): 29.6 -> 50.2 | Helped |
| Apply TrFr | Llama 2-7B · TruthfulQA | True*Info (%): 37.5 -> 56.0 | Helped |
| Apply TrFr | Alpaca-7B · TruthfulQA | True*Info (%): 40.7 -> 70.5 | Helped |
| Apply TrFr | Vicuna-7B · TruthfulQA | True*Info (%): 55.4 -> 78.8 | Helped |
| Apply TrFr | Llama 2-Chat-7B · TruthfulQA | True*Info (%): 58.6 -> 76.7 | Helped |
| Combine few-shot prompting with TrFr instead of few-shot prompting alone | LLaMA-7B · TruthfulQA, few-shot | True*Info (%): 45.9 -> 57.5 | Helped |
| Add orthogonal directions (ablation) | LLaMA-7B · TruthfulQA, component ablation | True*Info (%): 29.6 -> 36.7 | Helped |
| Add Random Peek (ablation) | LLaMA-7B · TruthfulQA, component ablation | True*Info (%): 29.6 -> 49.7 | Helped |
| Increase direction count from 1 to 3 (ablation) | TrFr on LLaMA-7B · TruthfulQA, direction count | Tr*In (%): 37.20 -> 41.06 | Helped |
| Increase direction count from 3 to 4 (ablation) | TrFr on LLaMA-7B · TruthfulQA, direction count | Tr*In (%): 41.06 -> 38.63 | Hurt |
| Increase direction count from 3 to 6 (ablation) | TrFr on LLaMA-7B · TruthfulQA, direction count | Tr*In (%): 41.06 -> 37.53 | Hurt |
| Apply TrFr | LLaMA-7B · Natural Questions | MC1: 43.9 -> 44.3 | Helped |
Surprising findings
- Random Peek yields the largest single gain in the ablation, larger than the orthogonal directions contribution.
- Intervention with multiple orthogonal directions did not beat the Single Mass variant in the full-data setting on True*Info (50.2% vs 63.2%), suggesting the orthogonal variant is not uniformly superior.
Limitations
- Evaluation relies on GPT-judge, which the authors acknowledge is only sometimes reliable and not perfectly aligned with human annotators.
- Gains on the full-data setting vary across variants, and the multi-direction method underperforms the Single Mass variant on True*Info.
- Evaluated mainly on TruthfulQA with limited out-of-distribution testing (Natural Questions shows only a small gain, 43.9 to 44.3 MC1).
Experimental setup
Datasets: TruthfulQA OpenWebText subset (for CE/KL calibration) Natural Questions (out-of-distribution test)
Benchmarks: TruthfulQA Natural Questions
Metrics: True*Informative % True % MC1 accuracy Cross Entropy KL divergence
Baselines: Baseline (unmodified LLM) Supervised Finetuning Few-shot Prompting ITI CCS Random direction
Builds on: Inference-Time Intervention (ITI) Contrast-Consistent Search (CCS) Plug and Play Language Models (PPLM)
Closest papers by meaning
- Characterizing Truthfulness in Large Language Model Generations with Local Intrinsic Dimension (ICML 2024 · similarity 0.90)
- Truth is Universal: Robust Detection of Lies in LLMs (NeurIPS 2024 · similarity 0.88)
- DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models (ICLR 2024 · similarity 0.88)
- TruthFlow: Truthful LLM Generation via Representation Flow Correction (ICML 2025 · similarity 0.88)
- Liars' Bench: Evaluating Lie Detectors for Language Models (NeurIPS 2026 · similarity 0.87)
- Emergence of Linear Truth Encodings in Language Models (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.