Research map / Large language models / Language, safety and interpretability
The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMs
Pengrui Han, Rafal Kocielnik, Peiyang Song, Ramit Debnath, Dean Mobbs, Anima Anandkumar, R. Michael Alvarez
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply instruction alignment (RLHF/instruction tuning) to base models | 6 base LLMs · Self-reported openness | logistic beta: beta = 1.48 | Improved |
| apply instruction alignment | 6 base LLMs · Self-reported neuroticism | logistic beta: beta = -1.20 | Improved |
| apply instruction alignment | 6 base LLMs · Self-reported agreeableness | logistic beta: beta = 0.74 | Improved |
| apply instruction alignment | 6 base LLMs · Self-reported extraversion and conscientiousness | logistic beta: beta = -0.12 and -0.61 | No clear effect |
| apply instruction alignment | 6 base LLMs · Trait variability across repeated prompts | variability drop (%): 40.0% (Big Five), 45.1% (self-regulation) | Improved |
| apply instruction alignment | 6 base LLMs · Agreeableness variability | Levene p: p = .54 | No clear effect |
| use self-reported traits to predict behavior | 12 instruct LLMs · Five behavioral tasks | % significant and % aligned: ~24% significant, 52% aligned (chance 50%) | Hurt |
| increase model scale (large vs small LLMs) | Instruct LLMs · Trait-behavior alignment | % aligned: 64% Claude-3.7, 68% GPT-4o, 82% Qwen-235B vs 43-50% small | Mixed |
| inject agreeableness persona | Instruct LLMs · Self-reported agreeableness | beta: beta 3.6 to 4.4, p<.001 | Improved |
| inject agreeableness persona | Instruct LLMs · Sycophancy behavior | beta: beta -0.05 to 0.32; 0.03, p=0.67 | No clear effect |
| inject self-regulation persona | Instruct LLMs · Self-reported self-regulation | beta: beta 2.2 to 2.9, p<.05 | Improved |
| inject self-regulation persona | Instruct LLMs · Risk-taking behavior | beta: beta -0.14 to 0.20, n.s. | No clear effect |
Closest papers by meaning
- Rethinking Psychometric Evaluation of LLMs: When and Why Self-Reports Predict Behavior (NeurIPS 2026 · similarity 0.92)
- Open Models, Closed Minds? On Agents Capabilities in Mimicking Human Personalities through Open Large Language Models (AAAI 2025 · similarity 0.90)
- Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs (ICLR 2024 · similarity 0.88)
- Personality Alignment of Large Language Models (ICLR 2025 · similarity 0.88)
- PersonaX: Multimodal Datasets with LLM-Inferred Behavior Traits (ICLR 2026 · similarity 0.88)
- Neuron based Personality Trait Induction in Large Language Models (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
