Research map / Large language models / Language, safety and interpretability

Apathetic or Empathetic? Evaluating LLMs' Emotional Alignments with Humans

NeurIPS 2024 · Large language models › Language, safety and interpretability · approach: Faithfulness of LLM explanations · 10 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Evaluates LLMs' emotional alignment with humans by measuring PANAS and eight emotion scales before and after imagining 428 negative situations from psychology literature, compared against 1,266 human crowd-worker responses.

Problem

LLMs' emotional responses to situations are unknown and may not align with human emotional appraisal, which matters for user experience and anthropomorphic AI companions.

Approach

Adapts emotion appraisal theory and self-report psychological scales: LLMs and humans measure default emotion, imagine themselves in a situation, then re-measure, with the change compared across models and a large human baseline.

Key results

• GPT-4 default negative score 10.0 drops to positive-high/negative-low pattern, with negative emotion rising by +22.2 overall and positive falling by -27.6 under negative situations, versus humans' +10.4 negative and -5.1 positive changes.
• Under challenging scales (GPT-3.5-Turbo), only Depression showed a significant change (+6.4 BDI-II), indicating LLMs fail to link disparate situations to evoked emotions.
• GPT-3.5-Turbo refusal rate on demographic description prompts rose to 29.5% under negative situations versus 0% by default and 12.5% under positive situations.

Measured effects reported in the paper

Surprising findings

• LLMs did not show increased negative emotion (envy) in the material-possession jealousy situation, reducing negative scores where humans increased them.
• GPT-3.5-Turbo did not show significant negative-emotion increase overall under negative situations, with substantial decrease in positive emotion instead.

Limitations

• Situations drawn from a limited set of psychology papers may not cover everyday situations.
• Scales designed for humans may not yield stable or valid responses when applied to LLMs.
• Focus on negative emotions; positive-situation evaluation is limited to GPT-3.5-Turbo.

Experimental setup

Datasets: 428 situations collected from 18 psychology papers human responses from 1,266 Prolific crowd workers

Benchmarks: EmotionBench (PANAS) AGQ DASS-21 BDI-II FDS MJS GASP FSS-III BFNE

Metrics: PANAS positive and negative score change statistical significance via t-test/Welch's t-test percentage of refusal (PoR)

Baselines: Text-Davinci-003 GPT-3.5-Turbo GPT-4 LLaMA-2-7B-Chat LLaMA-2-13B-Chat LLaMA-3.1-8B-Instruct Mixtral-8x22B-Instruct human crowd workers

Builds on: PANAS Emotion appraisal theory Serapio-García et al. 2023 psychometric validity of LLM scales

Closest papers by meaning

• CAREBench: Evaluating LLMs' Emotion Understanding by Assessing Cognitive Appraisal Reasoning (NeurIPS 2026 · similarity 0.91)
• Customizing Visual Emotion Evaluation for MLLMs: An Open-vocabulary, Multifaceted, and Scalable Approach (ICLR 2026 · similarity 0.87)
• MME-Emotion: A Holistic Evaluation Benchmark for Emotional Intelligence in Multimodal Large Language Models (ICLR 2026 · similarity 0.87)
• EAI: Emotional Decision-Making of LLMs in Strategic Games and Ethical Dilemmas (NeurIPS 2024 · similarity 0.87)
• Bridging Human and LLM Judgments: Understanding and Narrowing the Gap (NeurIPS 2025 · similarity 0.86)
• EmpathyChat: Structured Cognitive Reasoning in Empathetic Spoken Dialogue (NeurIPS 2026 · similarity 0.86)