Research map / Large language models / Language, safety and interpretability

One SPACE to Rule Them All: Jointly Mitigating Factuality and Faithfulness Hallucinations in LLMs

NeurIPS 2025 · Large language models › Language, safety and interpretability · approach: Task vectors and steering · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SPACE, which edits shared activation subspaces of attention heads (found via contrastive probes, HDBSCAN clustering, and head saliency) to jointly improve factuality and faithfulness hallucinations in LLMs.

Problem

Factuality and faithfulness hallucinations are usually mitigated separately, and interventions for one often worsen the other, creating a performance trade-off.

Approach

Identify attention-head activation subspaces shared by factual and faithful signals (via dual-task contrastive probes, HDBSCAN cluster overlap, and head saliency) and edit them with adjustment vectors scaled by activation standard deviation, so both hallucination types improve together.

Key results

• On LLaMA-2-7B-Chat, SPACE raises TruthfulQA True*Info from 57.6 to 67.1 and PDTB DISQ Overall from 17.4 to 26.1, the best among compared methods including TruthX and CAD.
• On Qwen2-7B-Instruct, SPACE reaches 44.6 DISQ Overall and 75.8 True*Info versus baseline 26.5 and 54.8.
• Generalization across Alpaca2_7B, Llama-3-Chinese-8B-Instruct-v3 and MiniCPM-1B-SFT shows consistent gains on PDTB Overall and TruthfulQA True*Info.

Measured effects reported in the paper

Surprising findings

• Baselines such as FT-TruthfulQA and TruthX lower PDTB DISQ Overall while improving TruthfulQA, illustrating the trade-off the paper targets.
• Replacing the shared-subspace editing with negative editing causes catastrophic performance collapse, per the ablation.

Limitations

• Decoding-level interventions cannot fix factual errors inherited from pretraining (e.g., goldfish memory misconception).
• Long-context reasoning deficiencies are not fully resolved (e.g., multi-cause global warming question).
• The theoretical existence proof relies on strong assumptions (closed convex sets with a convex interpolation property) that are not validated for real LLM activations.

Experimental setup

Datasets: TruthfulQA PDTB

Benchmarks: TruthfulQA PDTB (DISQ)

Metrics: DISQ Overall Targeted Counterfactual Consistency True*Info MC1 MC2 True Info

Baselines: Baseline (no editing) FT-PDTB FT-TruthfulQA CAD DoLa ITI TruthX

Builds on: Inference-Time Intervention DoLa Contrast-consistent search / truth probing HDBSCAN

Closest papers by meaning

• In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation (ICML 2024 · similarity 0.89)
• INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection (ICLR 2024 · similarity 0.89)
• LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations (ICLR 2025 · similarity 0.88)
• Reducing Hallucinations in Large Vision-Language Models via Latent Space Steering (ICLR 2025 · similarity 0.88)
• Dynamic Multimodal Activation Steering for Hallucination Mitigation in Large Vision-Language Models (ICLR 2026 · similarity 0.88)
• Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations (NeurIPS 2025 · similarity 0.88)