Research map / Large language models / Language, safety and interpretability
Generative Social Choice: The Next Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends generative social choice with costs/budgets and approximate LLM queries, giving approximate proportionality guarantees, and presents PROSE, a GPT-4o-based slate generator that beats four baselines on drug-review and city-deliberation datasets.
Problem
Producing a concise, budget-constrained slate of statements that proportionally represents diverse user opinions, where the candidate set is all possible statements and is accessible only through imperfect LLM queries.
Approach
Extends generative social choice to statements with variable costs (word lengths) and a total word budget, and proves cost-BJR guarantees that degrade gracefully under approximate discriminative and generative queries (errors beta, gamma, delta, mu); the PROSE implementation uses GPT-4o for both query types.
Key results
- PROSE achieves mean CoT utility 3.42 / 3.87 / 3.39 / 3.54 across the four instances, outperforming all baselines; improvements of roughly 10% to 40% in mean utility.
- Around 40% improvement in 25th-percentile agent utility over baselines.
- PROSE slates show 7 to 49 percentage points fewer cBJR violations than baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PROSE's budgeted, variable-length slate generation instead of unit-cost statements | PROSE · Birth Control (Uniform) | mean CoT utility: 3.05 -> 3.42 | Helped |
| Use PROSE's budgeted, variable-length slate generation instead of clustering-based slate generation | PROSE · Birth Control (Uniform) | fraction of sampled statements constituting a cBJR violation: 0.30 -> 0.19 | Helped |
| Use PROSE's budgeted, variable-length slate generation instead of clustering-based slate generation | PROSE · Birth Control (Imbalanced) | mean CoT utility: 3.28 -> 3.87 | Helped |
| Use PROSE's budgeted, variable-length slate generation instead of clustering-based slate generation | PROSE · Birth Control (Imbalanced) | fraction of sampled statements constituting a cBJR violation: 0.23 -> 0.14 | Helped |
| Use PROSE's budgeted, variable-length slate generation instead of unit-cost statements | PROSE · Obesity | mean CoT utility: 3.20 -> 3.39 | Helped |
| Use PROSE's budgeted, variable-length slate generation instead of clustering-based slate generation | PROSE · Obesity | fraction of sampled statements constituting a cBJR violation: 0.17 -> 0.10 | Helped |
| Use PROSE's budgeted, variable-length slate generation instead of zero-shot prompting with agent descriptions | PROSE · Bowling Green | mean CoT utility: 3.06 -> 3.54 | Helped |
| Use PROSE's budgeted, variable-length slate generation instead of zero-shot prompting with agent descriptions | PROSE · Bowling Green | fraction of sampled statements constituting a cBJR violation: 0.13 -> 0.05 | Helped |
Surprising findings
- PROSE-UnitCost, which produces slates 2-3x longer than the budget, performs worse than the budgeted PROSE.
- Simple Fast algorithm performs similarly to Complex in synthetic experiments, and all processes beat the worst-case theoretical guarantees.
Limitations
- GPT-4o query answers come without guarantees and can be hallucinated, prompt-dependent, or biased.
- Generative query degrades when queried on many agents with diverse opinions.
- Evaluation uses GPT-4o chain-of-thought utilities as proxy for true preferences rather than human ratings.
Experimental setup
Datasets: UCI ML Drug Review dataset Bowling Green deliberation dataset (Polis)
Benchmarks: UCI ML Drug Review subsampled instances (Birth Control Balanced, Birth Control Imbalanced, Obesity) Bowling Green city deliberation instance
Metrics: mean agent utility (CoT utility 1-6) 25th percentile utility fraction of sampled statements violating cBJR p-value vs PROSE
Baselines: PROSE-UnitCost Clustering Zero-Shot Contextless Zero-Shot
Builds on: generative social choice balanced justified representation extended justified representation participatory budgeting
Closest papers by meaning
- SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Models (AAAI 2024 · similarity 0.85)
- Group Preference Optimization: Few-Shot Alignment of Large Language Models (ICLR 2024 · similarity 0.85)
- Social Choice Foundations for Simulation-Augmented Generation (NeurIPS 2026 · similarity 0.84)
- Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference (ICML 2024 · similarity 0.84)
- No Preference Left Behind: Group Distributional Preference Optimization (ICLR 2025 · similarity 0.84)
- Prompting Fairness: Integrating Causality to Debias Large Language Models (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.