Research map / Large language models / Language, safety and interpretability
Truth is Universal: Robust Detection of Lies in LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that LLM activations for true/false statements lie in a two-dimensional truth subspace (general truth direction t_G plus polarity-sensitive t_P) and proposes TTPD, a lie detector trained on this structure that reaches about 94% accuracy on real-world lies.
Problem
Activation-based lie detectors trained on LLM internals fail to generalise, for example from affirmative to negated statements, and it was unclear whether a single general truth direction exists.
Approach
Identify a two-dimensional truth subspace spanned by a general truth direction t_G and a polarity-sensitive direction t_P, learned by least squares with topic-mean centering and balanced polarity; then train TTPD, a logistic regression on the projections onto t_G and a learned polarity direction p.
Key results
- TTPD reaches 93.9 ± 0.2% average accuracy on unseen topics, comparable to LR (94.6 ± 0.7%) and above CCS (84.8 ± 6.4%) and MM (92.2 ± 0.4%).
- On real-world role-play lies, TTPD classifies unambiguous lies and truthful replies with 93.8 ± 1.5% accuracy, versus MM 90.5 ± 1.5%, LR 79 ± 8%, and CCS 73 ± 12%.
- The two-dimensional truth subspace is found across LLaMA3-8B, Gemma-7B, Gemma-2-27B, LLaMA2-13B, and Mistral-7B; the first two PCs of preprocessed activations explain more than 60% of variance and align with t_G and t_P (cosine 0.98 and 0.97).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TTPD instead of Logistic Regression, CCS, or Mass Mean | LLaMA3-8B-Instruct lie detection · Real-world lies | accuracy: 93.8 \pm 1.5\% vs. 90.5 \pm 1.5\%, 79 \pm 8\%, and 73 \pm 12\% | Helped |
| Use TTPD instead of Logistic Regression, CCS, or Mass Mean | LLaMA3-8B-Instruct lie detection · Unseen topics | accuracy: 93.9 \pm 0.2\% vs. 94.6 \pm 0.7\%, 84.8 \pm 6.4\%, and 92.2 \pm 0.4\% | Mixed |
| Train the truth direction on affirmative and negated statements instead of affirmative statements alone (ablation) | Linear truth direction · Negated statements | AUROC | Helped |
| Add logical conjunctions and disjunctions to affirmative and negated training statements (ablation) | Linear truth direction · Conjunctions and disjunctions | AUROC | No clear effect |
| Project training activations onto the orthogonal complement of the two-dimensional truth subspace (ablation) | Linear truth direction trained on cities · Other topics | AUROC | Hurt |
| Add logical conjunctions to training after projecting out the two-dimensional truth subspace (ablation) | Linear truth direction trained on cities · Conjunctions and disjunctions on other topics | AUROC | Helped |
| Use the general truth direction instead of the affirmative truth direction | Linear truth classification · Negated statements | AUROC: 0.98 for the general truth direction | Helped |
| Use TTPD instead of Logistic Regression | LLaMA3-8B-Instruct lie detection · Ambiguous lies | accuracy: 59 \pm 3\% vs. 68 \pm 16\% | Hurt |
Surprising findings
- Affirmative-only truth direction t_A is a linear combination of t_G and t_P, which explains why it fails to generalise to negated statements.
- TTPD generalises to real-world lies despite being trained only on simple factual statements, while the LR baseline's truth direction generalised worse on real-world lies.
Limitations
- TTPD uses only one of the two truth-subspace dimensions; a nonlinear classifier over both might do better.
- Generalisation was tested on a limited set of statement types and real-world scenarios, and real-world labels rely on manual categorisation by one author.
- Universality claim is limited to at least two dimensions; other statement types may reveal additional structure.
Experimental setup
Datasets: cities sp_en_trans element_symb animal_class inventors facts neg_ variants conj and disj variants German translations common_claim_true_false counterfact_true_false larger_than
Benchmarks: cities neg_cities sp_en_trans element_symb animal_class inventors facts conj/disj statements German statements common_claim_true_false counterfact_true_false real-world role-playing lie scenarios
Metrics: accuracy AUROC
Baselines: Logistic Regression Contrast Consistent Search Mass Mean probe
Builds on: Marks and Tegmark geometry of truth contrast consistent search mass mean probing logistic regression probing
Closest papers by meaning
- How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions (ICLR 2024 · similarity 0.94)
- Liars' Bench: Evaluating Lie Detectors for Language Models (NeurIPS 2026 · similarity 0.92)
- Truth Forest: Toward Multi-Scale Truthfulness in Large Language Models through Intervention without Tuning (AAAI 2024 · similarity 0.88)
- LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations (ICLR 2025 · similarity 0.88)
- Characterizing Truthfulness in Large Language Model Generations with Local Intrinsic Dimension (ICML 2024 · similarity 0.88)
- HonestLLM: Toward an Honest and Helpful Large Language Model (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.