Research map / Large language models / Language, safety and interpretability

Trust, but Don’t Verify: Epistemic Blind Spots in LLM Source Evaluation

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: Statistical frameworks for LLM evaluation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that LLMs detect fabricated statistics in isolation but fail to use that capability during multi-source synthesis, trusting impossible statistics as much as valid ones, with mechanistic evidence for a consensus-gated methodology-register shortcut.

Problem

Whether LLMs evaluate the quality of evidence from multiple sources or merely aggregate it by surface presentation, especially when sources contain fabricated but methodologically credible-looking statistics.

Approach

A factorial experiment in four-source group threads shows that a methodology-register gate drives source influence while numeric validity is ignored, even though the same models flag the fabrication in isolation; probes and causal tracing localize the failure to a consensus-gated methodology representation with no numeric-validity correction.

Key results

• Impossible statistics recover about 74% of the influence shift produced by valid statistics on the focal source's estimate across models.
• Methodology cross-domain probe AUC is about 0.83-0.92 at layer 8 while numerics probes stay at chance; numerics separability in isolation is about 0.87-0.88 and collapses to about 0.52-0.56 under synthesis.
• Oracle checklist prompting reduces trust in impossible statistics but also reduces trust in valid statistics, producing blanket skepticism rather than selective discernment.

Surprising findings

• Models flag impossible statistics in their own reasoning yet still endorse the source at high rates.
• An oracle checklist naming the exact anomaly produces blanket skepticism rather than selective discernment.

Limitations

• Fixed four-source structure limits ecological validity.
• Mechanistic analysis limited to Qwen 32B with partial replication on OLMo 3.1 and pre-reasoning position.
• Dual-process interpretation is interpretive and not directly tested.

Experimental setup

Benchmarks: factorial multi-source synthesis trials (VC, MKT, PH)

Metrics: source preference index correct identification rate probe AUC causal restore fraction direct logit attribution focal source share

Baselines: Claude Haiku 4.5 Claude Sonnet 4.5 Claude Opus 4.5 Qwen 32B OLMo 3.1 Think GPT-5.4

Builds on: sycophancy evaluation judge-advisor weight-of-advice ROME causal tracing difference-of-means probes RLHF DPO

Closest papers by meaning

• Information Discernment in Large Language Models (NeurIPS 2026 · similarity 0.90)
• Failing to Falsify: Evaluating and Mitigating Confirmation Bias in Language Models (NeurIPS 2026 · similarity 0.87)
• LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations (ICLR 2025 · similarity 0.87)
• PRISMM-Bench: A Benchmark of Peer-Review Grounded Multimodal Inconsistencies (ICLR 2026 · similarity 0.87)
• LLMs Show No Signs Of Individuated Metacognition (NeurIPS 2026 · similarity 0.86)
• To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contexts (ICLR 2025 · similarity 0.86)