Research map / Large language models / AI and society

LLM Generated Persona is a Promise with a Catch

NeurIPS 2025 · Large language models › AI and society · approach: Critiques of LLM claims and judging · 10 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that LLM-generated personas (especially descriptive ones with more LLM-written content) produce biased, non-representative opinion simulations, e.g., predicting a Democratic sweep in the 2024 US election, and open-sources about one million personas.

Problem

Persona-based LLM simulation of population opinions is increasingly used as a cheap substitute for human surveys, but current LLM-based persona generation methods are ad hoc and may produce biased, unrepresentative populations.

Approach

Systematizes four tiers of persona generation (census-based Meta personas, Objective and Subjective Tabular personas, and free-form Descriptive personas) and shows that increasing the amount of LLM-generated persona content shifts simulated opinions leftward and away from real outcomes, with bias attributed to the positive, subjective tone of LLM-written personas.

Key results

• Using Llama 3.1 70B for both persona generation and simulation, Descriptive personas drive simulated 2024 election results toward a Democratic sweep across all states, while Meta personas yield distributions closest to reality.
• Across roughly 1,000,000 generated personas from six open-source LLMs and 500+ questions, the leftward bias is reported as universal across simulation models rather than model-specific.
• On OpinionQA, the paper reports that alignment score variance across persona types differs by topic, and LLM-generated content shifts opinions from traditional to progressive.

Measured effects reported in the paper

Surprising findings

• Simulated 2016 and 2020 election results also drift leftward despite these outcomes likely being in training data.
• Yi-34B Chat is an exception, showing a bias toward right-leaning viewpoints in the 2020 election simulation.

Limitations

• Persona-level bias analysis relies on a limited set of open-source models and one simulation prompt design.
• Descriptive personas' bias is shown qualitatively for non-political domains without ground truth.
• Proposed directions (calibration, benchmarks) are not implemented in the paper.

Experimental setup

Datasets: U.S. Census Bureau joint distributions (Age, Sex, Race, State) OpinionQA (Pew Research surveys) 2016, 2020, 2024 US presidential election results

Benchmarks: 2016, 2020, 2024 US presidential elections OpinionQA (first 500 questions)

Metrics: alignment score (1 - Wasserstein distance) sentiment polarity subjectivity (TextBlob)

Baselines: Meta Personas Objective Tabular Personas Subjective Tabular Personas Descriptive Personas

Builds on: OpinionQA silicon sampling

Closest papers by meaning

• Persona Generators: Generating Diverse Synthetic Personas for Arbitrary Contexts (NeurIPS 2026 · similarity 0.92)
• When Can Digital Personas Reliably Approximate Human Survey Findings? (NeurIPS 2026 · similarity 0.89)
• Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs (ICLR 2024 · similarity 0.88)
• Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Traces (NeurIPS 2026 · similarity 0.87)
• Position: LLM Social Simulations Are a Promising Research Method (ICML 2025 · similarity 0.87)
• PersonalLLM: Tailoring LLMs to Individual Preferences (ICLR 2025 · similarity 0.87)