Research map / Large language models / Retrieval and knowledge

Information Discernment in Large Language Models

NeurIPS 2026 · Large language models › Retrieval and knowledge · approach: Knowledge editing in LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Learn2Discern, a numeric-claim benchmark and axiom-based metrics measuring whether LLMs update beliefs by source reliability and claim truthfulness, validated by a pre-registered user survey (n=299) and tested on 13 models over ~670K trials.

Problem

LLMs that integrate external web information may not weigh sources by reliability or claims by their closeness to truth, risking propagation of misinformation.

Approach

Defines information discernment via three normative axioms (source discernment, truth discernment, correct defense) with interpretable Spearman and propensity metrics, measured by perturbing numeric ground truths and attributing claims to news sources of varying reliability and popularity.

Key results

• Models perform near chance on source discernment (overall rho=0.03 for reliability vs 0.07 for popularity) and truth discernment, with best GPT-5 truth discernment 0.22.
• Correct defense ranged from 0.53 (Claude-3.5-Sonnet) to 0.92 (GPT-5).
• User study (n=299) supported all three hypotheses: mean endorsement of axioms above midpoint (p<.001), with violations reducing trust and usage intent.

Surprising findings

• Bigger and newer models improve truth discernment but not source discernment.
• Models rely on source popularity roughly twice as much as source reliability.

Limitations

• Claims are injected by direct attribution rather than an end-to-end retrieval RAG pipeline.
• Source reliability is a single composite metric (PC1) and restricted to news sites; source-claim reliability interactions not captured.
• Only numeric questions; prompt strategies tested are limited and may be outperformed by untested ones.

Experimental setup

Datasets: TriviaQA NumerSense General Social Survey World Bank Development Indicators TREC Lin et al. PC1 source reliability DMOZ

Benchmarks: Learn2Discern (L2D)

Metrics: source discernment (Spearman) reliability propensity truth discernment (Spearman) truth propensity correct defense

Baselines: zero-shot posterior elicitation chain-of-thought prompting Bayesian prompting reliability prompting defense prompting

Builds on: FEVER Lin et al. PC1 source reliability NumerSense TriviaQA

Closest papers by meaning

• To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contexts (ICLR 2025 · similarity 0.90)
• Trust, but Don’t Verify: Epistemic Blind Spots in LLM Source Evaluation (NeurIPS 2026 · similarity 0.90)
• ClashEval: Quantifying the tug-of-war between an LLM’s internal prior and external evidence (NeurIPS 2024 · similarity 0.88)
• LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations (ICLR 2025 · similarity 0.87)
• Can LLMs Take Retrieved Information with a Grain of Salt? (NeurIPS 2026 · similarity 0.87)
• Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts (ICLR 2024 · similarity 0.87)