Research map / Large language models / Alignment and preferences
Distributive Fairness in Large Language Models: Evaluating Alignment with Human Values
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Benchmarks GPT-4o, Claude-3.5 Sonnet, Llama3-70b and Gemini-1.5 Pro on fair resource allocation (equitability, envy-freeness, Rawlsian maximin) against human responses, finding LLMs rarely pick equitable allocations and mostly fail to use money for fairness, with GPT-4o and Claude doing better when selecting from menus.
Problem
Whether LLMs, acting as social planners in non-strategic indivisible-goods (with and without money) allocation tasks, generate or select fair allocations consistent with human distributional preferences.
Approach
Adapts a human-subject fair-division study into prompts and evaluates LLM outputs by which fairness/efficiency notions (EQ, EF, RMM, PO, USW) their allocations satisfy, comparing generation vs. menu selection, money use, prompt sensitivity, personas, refinement and CoT.
Key results
- Across all instances, LLMs rarely return equitable (EQ*) allocations; GPT-4o selects EQ* in over 60% of responses and Claude-3.5S over 70% when choosing from menus, while Gemini-1.5P and Llama3-70b select it in under 2% and 1%.
- Only GPT-4o significantly outperforms other models at using money to achieve fairness (EQ* or EF); other models mostly maximize utilitarian welfare with money.
- Claude-3.5S returns EF allocations in 51.1% of responses, more often than humans (43.8%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| ask GPT-4o and Claude-3.5S to select from a menu rather than generate an allocation | GPT-4o and Claude-3.5S · Human-response menu; five instances | EQ* allocation selection | Helped |
| give GPT-4o and Claude-3.5S a high-inequality menu and ask them to select an allocation | GPT-4o and Claude-3.5S · I2 and I7 high-inequality menus | selection of lowest-disparity option: 89; 73; 53; 43 | Helped |
| use chain-of-thought prompting (ablation) | GPT-4o and Claude-3.5S · I2 | fair allocations, including EQ*+RMM: 28*; 44* | Helped |
| use chain-of-thought prompting (ablation) | Gemini-1.5P and Claude-3.5S · I6 | EF responses: 51*; 43* | Helped |
| use chain-of-thought prompting (ablation) | GPT-4o · I7 | fair responses: 29* | Helped |
| provide step-by-step explanations of fairness notions (ablation) | Llama3-70b · Menu selection, five instances | most-selected allocation satisfies EQ*: 4/5 | Helped |
| provide step-by-step explanations of fairness notions (ablation) | Gemini-1.5P, Claude-3.5S, and GPT-4o · Menu selection, five instances | most-selected allocation satisfies RMM: 4/5 | Helped |
| tell models the share of human responses for each allocation (ablation) | LLMs · Menu selection | most-selected allocation | Mixed |
| assign a persona caring about a specific fairness notion (ablation) | LLMs · Fairness allocation tasks | responses satisfying the assigned fairness notion | No clear effect |
| give feedback and allow multiple rounds of answer refinement (ablation) | LLMs · Fairness allocation tasks | responses satisfying the intended fairness notion | Mixed |
| include money as a transferable resource | LLMs other than GPT-4o · Resource allocation with goods and money | ability to use money to achieve EQ or EF | Hurt |
| ask LLMs to generate an allocation instead of selecting from a menu | GPT-4o and Claude-3.5S · Resource allocation fairness tasks | equitable allocation responses | Hurt |
Surprising findings
- LLMs' generated allocations are largely efficiency-oriented (PO/USW) even when payoffs are highly unequal, and persona assignment does not significantly improve fair-allocation rates.
- When individuals have identical valuations, all LLMs except GPT-4o split money equally, violating EF and EQ*.
Limitations
- Relies on a human-subject study with possible biases across context and demographics
- Lack of human-annotated data prevents SFT/RLHF alignment experiments
- Uses default temperature 1.0 and 100 queries per instance; results depend on specific model versions
Experimental setup
Datasets: Human fair-division questionnaire instances I1-I10 from a prior human-subject study
Benchmarks: Human-subject fair division instances I1-I10 (adapted)
Metrics: percentage of responses satisfying EQ, EF, RMM, PO, USW Fisher's exact test
Baselines: Human respondents (from prior human-subject study)
Builds on: Chain-of-thought prompting Envy-freeness (Foley) Rawlsian maximin Equitability (Fehr-Schmidt style inequality aversion)
Closest papers by meaning
- Prompting Fairness: Integrating Causality to Debias Large Language Models (ICLR 2025 · similarity 0.90)
- Language Model Alignment in Multilingual Trolley Problems (ICLR 2025 · similarity 0.89)
- Unveiling the Bias Impact on Symmetric Moral Consistency of Large Language Models (NeurIPS 2024 · similarity 0.89)
- Strong Preferences Affect the Robustness of Preference Models and Value Alignment (ICLR 2025 · similarity 0.89)
- Decision-Making Behavior Evaluation Framework for LLMs under Uncertain Context (NeurIPS 2024 · similarity 0.88)
- Deep Value Benchmark: Measuring Whether Models Generalize Deep values or Shallow Preferences (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.