Research map / Large language models / AI and society

The Illusion of Empathy: How AI Chatbots Shape Conversation Perception

AAAI 2025 · Large language models › AI and society · approach: Data commons and human baselines · 19 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Compares user-reported empathy and conversation quality for GPT-based chatbots versus human partners across WASSA 2023/2024 and Empathic Conversations data, and checks agreement with GPT-4o judgments, a trained perceived-empathy model, and pre-trained empathy models.

Problem

Whether and how users perceive chatbots as less empathetic than humans, and how perceived empathy relates to overall conversation quality in human-chatbot conversations, has been under-explored with direct user feedback.

Approach

Uses user self-reports of empathy collected directly after human-chatbot and human-human conversations, and shows that chatbots get higher conversation quality ratings despite lower perceived empathy, with GPT-4o empathy annotations aligning with user ratings.

Key results

• Chatbots were rated significantly lower than humans on general empathy (4.04 vs 5.42) and state empathy, while closeness did not differ.
• Across mixed models, chatbot conversations had higher conversation quality (beta > 1.05, p < 0.01) than human conversations.
• GPT-4o rated chatbots as less empathetic than humans (mean 4.10 vs 4.98, p < 0.001) with correlation r = 0.20 for human ratings and r = 0.06 for chatbot ratings.

Measured effects reported in the paper

Surprising findings

• Chatbots were rated higher on conversation quality even though they were perceived as less empathetic.
• The pre-trained and perceived empathy models did not separate chatbots from humans on general empathy, and GPT-4o identified participants with only 61% accuracy.

Limitations

• No participant group was unaware they were interacting with a chatbot, so identity-awareness effects cannot be assessed.
• Participant pool is not representative of a global population (crowdsourced MTurk workers).
• Correlations between language-based empathy predictions and human ratings are weak (r around 0.06 to 0.20).

Experimental setup

Datasets: Empathic Conversations (EC) WASSA 2023 WASSA 2024

Benchmarks: WASSA 2023 WASSA 2024 Empathic Conversations

Metrics: t-test Pearson correlation mixed-effects model coefficients accuracy F1 score

Baselines: GPT-4o pre-trained empathy models (Batson empathy, Interpret, Emo-React, Explore)

Builds on: WASSA 2023 shared task WASSA 2024 shared task Empathic Conversations dataset DLATK

Closest papers by meaning

• EmpathyChat: Structured Cognitive Reasoning in Empathetic Spoken Dialogue (NeurIPS 2026 · similarity 0.87)
• RealityTest: How People Probe AI Identity and Whether Models Disclose It (NeurIPS 2026 · similarity 0.84)
• The Good, The Bad, and Why: Unveiling Emotions in Generative AI (ICML 2024 · similarity 0.83)
• First-Person Fairness in Chatbots (ICLR 2025 · similarity 0.83)
• MetaMind: Modeling Human Social Thoughts with Metacognitive Multi-Agent Systems (NeurIPS 2025 · similarity 0.83)
• Apathetic or Empathetic? Evaluating LLMs' Emotional Alignments with Humans (NeurIPS 2024 · similarity 0.83)