Research map / Large language models / Retrieval and knowledge
Citations and Trust in LLM Generated Responses
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Live QA experiment (N=303 Prolific participants) testing whether citations (0, 1, or 5; relevant or random) and citation checking affect self-reported trust in ChatGPT-4 responses, grounded in trust-as-anti-monitoring and social proof theory.
Problem
Whether citations in LLM-generated responses increase user trust, whether the number and relevance of citations matter, and whether checking citations reduces trust.
Approach
Applies the anti-monitoring theory of trust and the principle of social proof to a between-subjects live QA experiment, manipulating citation count and validity and tracking actual citation-checking behavior via mouse hovers.
Key results
- Presence of citations significantly increased trust (beta=0.394, p<0.01), while random citations reduced trust relative to valid citations (beta=-0.268, p<0.01).
- Five citations were not rated more trustworthy than one citation (p>0.05), both above zero citations.
- Checking citations was associated with lower trust (beta=-0.058, p<0.01); only 9.77% of answers with citations were checked.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| include citations rather than none | ChatGPT4 responses · ChatGPT4 responses | trust rating: +0.394 β | Helped |
| show random citations rather than accurate citations | ChatGPT4 responses · ChatGPT4 responses | trust rating: -0.268 β | Hurt |
| provide one citation rather than zero | ChatGPT4 responses · ChatGPT4 responses | trust rating mean: 7.732 -> 8.103; 0.371 | Helped |
| provide five citations rather than zero | ChatGPT4 responses · ChatGPT4 responses | trust rating mean: 7.732 -> 8.170; 0.438 | Helped |
| provide five citations rather than one | ChatGPT4 responses · ChatGPT4 responses | trust rating mean: 8.103 -> 8.170; 0.067 | No clear effect |
| check citations rather than not checking them | ChatGPT4 responses · ChatGPT4 responses | trust rating: -0.058 β | Hurt |
| ask a political rather than non-political question | ChatGPT4 responses · ChatGPT4 responses | trust rating: +0.314 β | Helped |
| ask a fact-based rather than non-fact-based question | ChatGPT4 responses · ChatGPT4 responses | trust rating: +0.374 β | Helped |
| ask a complex rather than other question | ChatGPT4 responses · ChatGPT4 responses | trust rating: 0.070 β | No clear effect |
| provide five citations rather than one | ChatGPT4 responses · citation-check likelihood | citation-check likelihood: 59 vs 24 checks | Helped |
Surprising findings
- Random citations increased trust relative to no citations, though the effect was not significant among checked answers.
- Five citations were not perceived as more trustworthy than a single citation.
Limitations
- Sample drawn entirely from Prolific and skewed (65% white, tech-savvy), limiting generalizability.
- Trust measured with a single one-item 1-10 rating.
- Question content was not controlled across conditions; underpowered for complex-question analysis.
Experimental setup
Datasets: AskReddit Quora
Metrics: self-reported trust rating (1-10) citation check rate (mouse hover)
Builds on: trust as anti-monitoring principle of social proof retrieval augmented generation
Closest papers by meaning
- Chain-of-Thought Improves Text Generation with Citations in Large Language Models (AAAI 2024 · similarity 0.86)
- Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews (ICML 2024 · similarity 0.85)
- Measuring and Enhancing Trustworthiness of LLMs in RAG through Grounded Attributions and Learning to Refuse (ICLR 2025 · similarity 0.85)
- Walk the Talk? Measuring the Faithfulness of Large Language Model Explanations (ICLR 2025 · similarity 0.84)
- To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contexts (ICLR 2025 · similarity 0.84)
- Position: TrustLLM: Trustworthiness in Large Language Models (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.