Research map / Large language models / Language, safety and interpretability

More Is Not More: What Matters for Diversity in LLM Opinions？

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: Bias measurement and mitigation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Factorial audit of LLM opinion diversity separating persona depth, interaction architecture, and low-cost tricks across 7 chat models on 100 WildChat questions, measured with MPD, cluster count, Vendi Score, and beta-diversity metrics.

Problem

LLM-simulated opinions are homogenized, and common diversity interventions (persona depth, multi-agent/multi-turn architectures, low-cost tricks) are evaluated in isolation with incomparable metrics and are typically upgraded together, so their individual contributions cannot be attributed.

Approach

A factorial design that independently varies persona depth (None to Pro) and interaction architecture (Single-Call, Multi-Turn, Multi-Agent), plus a unified opinion-extraction and embedding pipeline with alpha-diversity (MPD, cluster count, Vendi) and beta-diversity (beta-Vendi, unique cluster ratio) metrics.

Key results

• Role (single-sentence occupation) yields delta >= 0.62 over None on all seven models and captures most of the None-to-Pro MPD gain; Basic falls below Role on Gemini, Kimi and GPT-mini.
• Multi-Turn and Multi-Agent architectures have mean UCR of 79% between them, with ~50% excess UCR after split-half calibration, indicating largely non-overlapping opinion coverage.
• Trait Assignment is the strongest low-cost trick at about 37% of Role's effect, and a one-sentence persona outperforms the best low-cost trick by 2.5x.

Surprising findings

• More detailed demographic personas (Basic) can yield lower diversity than the occupation-only Role on some models.
• Multi-Agent discussion nearly eliminates the persona-driven gain in rarefied cluster count (+15% vs +46% under Single-Call), indicating mutual visibility compresses distinct opinions.

Limitations

• Multi-Agent condition uses only neutral prompts; debate or role-conflict configurations not tested.
• Only 100 questions focused on opinion-eliciting text domains and 20 personas, so demographic representativeness is not claimed.
• Does not vary training-time interventions such as fine-tuning or preference optimization; diversity measured in semantic embedding space without human opinion distribution calibration.

Experimental setup

Datasets: WildChat

Benchmarks: WildChat-derived 100 open-ended questions

Metrics: mean pairwise distance cluster count Vendi Score beta-Vendi Score unique cluster ratio Cliff's delta

Baselines: Single-Call with no persona Enhanced Prompt High Temperature Demographic Cueing Trait Assignment

Builds on: Vendi Score Wilcoxon signed-rank test Benjamini-Hochberg correction

Closest papers by meaning

• Can DPO Learn Diverse Human Values? A Theoretical Scaling Law (NeurIPS 2025 · similarity 0.89)
• Diversity as a Reward: Fine-Tuning LLMs on a Mixture of Domain-Undetermined Data (NeurIPS 2025 · similarity 0.88)
• Does Writing with Language Models Reduce Content Diversity? (ICLR 2024 · similarity 0.88)
• Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference (ICML 2024 · similarity 0.88)
• Bridging Human and LLM Judgments: Understanding and Narrowing the Gap (NeurIPS 2025 · similarity 0.87)
• SimpleStrat: Diversifying Language Model Generation with Stratification (NeurIPS 2025 · similarity 0.87)