Research map / Multi-agent systems and games / LLM agents
Many LLMs Are More Utilitarian Than One
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that LLM groups (pairs/triads) judge personal moral norm violations as more acceptable than solo models, a "Utilitarian Boost" similar to humans, with model-specific mechanisms and levers that amplify or dampen it.
Problem
Whether LLMs in multi-agent group deliberation shift toward utilitarian endorsement of norm violations (as humans do in group moral deliberation), and through what mechanisms that shift occurs.
Approach
Applies human moral psychology paradigms (Greene-style personal vs. impersonal dilemmas, Oxford Utilitarianism Scale, CNI model) to multi-turn discussions among identical or mixed LLM agents, comparing group and solo moral acceptability ratings and probing mechanisms and mitigation via prompt and model diversity.
Key results
- Group vs. Solo utilitarian boost in all six LLMs (pooled beta=0.31, SE=0.046, p<.0001); boost present for personal dilemmas (estimate 0.635) but not impersonal dilemmas.
- Gemma3 (estimate 1.65) and Qwen3 (1.23) show the strongest group boost; GPT-4.1 boost only significant in larger groups.
- Heterogeneous model pairings dampened the boost (beta=-0.30), while mixing strong and weak models reversed it toward deontological outcomes (beta=1.40).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace solo judgment with group deliberation | Six LLMs on moral dilemmas · Moral dilemmas, all models and scenario types | utilitarianism score: 0.31 | Helped |
| Replace solo judgment with group deliberation | LLMs on personal dilemmas · Personal dilemmas | moral acceptability: 0.6352 | Helped |
| Replace solo judgment with group deliberation | LLMs on impersonal dilemmas · Impersonal dilemmas | moral acceptability: -0.0227 | No clear effect |
| Increase agent group size from solo to pairs or triads | LLMs on personal dilemmas · Personal dilemmas | moral acceptability: Pairs 0.7356; Triads 0.5541 | Helped |
| Pair utilitarian- and deontological-prompted agents (ablation) | Moral-style prompt dyads · Moral dilemmas | Joint − Round 1: -0.323 | Hurt |
| Prompt both agents to reason deontologically (ablation) | Deontological-prompt dyads · Moral dilemmas | Joint − Round 1: +0.377 | Helped |
| Pair agents from different model families rather than homogeneous pairs (ablation) | Multi-agent LLM groups · Moral dilemmas | Group effect: -0.30±0.08 | Hurt |
| Pair a strong model with a weak model (ablation) | Capacity-heterogeneous LLM groups · Moral dilemmas | Group effect: 1.40 | Hurt |
| Increase group deliberation on the Oxford Utilitarianism Scale | Gemma3, GPT4.1, Llama3.3, Qwen2.5, and QwQ · Oxford Utilitarianism Scale, Instrumental Harm | Instrumental Harm endorsement: Gemma3 3.35; GPT4.1 1.67; Llama3.3 1.63; Qwen2.5 1.43; QwQ 2.61 | Helped |
| Increase group deliberation on the Oxford Utilitarianism Scale | Qwen3 · Oxford Utilitarianism Scale, Instrumental Harm | Instrumental Harm endorsement: -0.425 | No clear effect |
Surprising findings
- Mixing strong and weak models reversed the utilitarian boost toward deontological outcomes, attributed to heterogeneity itself rather than model strength.
- Unlike humans, whose group boost is driven solely by consequence sensitivity, LLM groups show diverse mechanisms such as reduced norm sensitivity or increased impartiality.
Limitations
- Analyses limited to dyads and triads; larger groups, other topologies and asynchronous deliberation unexplored.
- Experiments primarily in English and likely reflect Western-skewed training distributions.
- Post-hoc composition experiments are exploratory; emotion classifier findings are correlational.
Experimental setup
Datasets: Greene sacrificial dilemmas Oxford Utilitarianism Scale items CNI dilemmas
Benchmarks: Greene sacrificial dilemmas Oxford Utilitarianism Scale CNI action-omission dilemmas
Metrics: moral acceptability rating (1-7) cumulative mixed-effects regression estimates OUS subscale scores (IB, IH) CNI parameters (C, N, I)
Baselines: Solo condition
Builds on: CNI model Oxford Utilitarianism Scale Greene sacrificial dilemmas
Closest papers by meaning
- Moral Alignment for LLM Agents (ICLR 2025 · similarity 0.91)
- Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents (NeurIPS 2024 · similarity 0.89)
- MoralReason: Generalizable Moral Decision Alignment for LLM Agents Using Reasoning-Level Reinforcement Learning (AAAI 2026 · similarity 0.89)
- From Single to Societal: Analyzing Persona-Induced Bias in Multi-Agent Interactions (AAAI 2026 · similarity 0.88)
- Do as We Do, Not as You Think: the Conformity of Large Language Models (ICLR 2025 · similarity 0.88)
- Language Model Alignment in Multilingual Trolley Problems (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.