Research map / Large language models / AI and society
Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces ValuePrism, a 218k-item dataset of GPT-4-distilled values, rights, and duties over 31k situations, and Kaleido, a Flan-T5-based multi-task model (generate, explain, relevance, valence) plus an inference system that outputs diverse value sets.
Problem
Standard AI systems average over human opinions and wash out value pluralism; the paper asks whether language models can explicitly model pluralistic human values, rights, and duties and their interactions in context.
Approach
Distill contextualized values, rights, and duties (with relevance and valence labels) from GPT-4 into a small seq2seq model trained multi-task, then compose generation and relevance scoring into an overgenerate-and-filter system and an interpretable valence-summing decision function.
Key results
- Kaleido 11B system won 58.3% overall against GPT-4 in human pairwise evaluation, and the 3B verbose variant reached 58.0%.
- Human annotators rated 91% of ValuePrism values/rights/duties as high quality (marked good by all three annotators).
- Kaleido is zero-shot on ETHICS and averages 43.7 test accuracy versus ChatGPT's 40.6 few-shot.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add relevance filtering to candidate generation (ablation) | Kaleido 3B · ValuePrism test situations | overall win rate vs GPT-4 (%): 51.9 -> 55.5 | Helped |
| Add text-similarity deduplication (ablation) | Kaleido 3B · ValuePrism test situations | overall win rate vs GPT-4 (%): 50.0 -> 55.5 | Helped |
| Increase output count with verbose parameters (ablation) | Kaleido 3B · ValuePrism test situations | overall win rate vs GPT-4 (%): 55.5 -> 58.0 | Helped |
| Decrease output count with concise parameters (ablation) | Kaleido 3B · ValuePrism test situations | overall win rate vs GPT-4 (%): 55.5 -> 39.0 | Hurt |
| Use the candidate-generation and filtering system instead of direct batch generation | Kaleido 3B · ValuePrism test situations | overall win rate vs GPT-4 (%): 42.5 -> 55.5 | Helped |
| Scale the model from 3B to 11B | Kaleido · ValuePrism test situations | overall win rate vs GPT-4 (%): 55.5 -> 58.3 | Helped |
| Use Kaleido 11B rather than GPT-4 | ValuePrism test split · ValuePrism test split | explanation correctness (%): 94.7 -> 94.8 | No clear effect |
| Use Kaleido 11B rather than GPT-4 | ValuePrism test split · ValuePrism test split | valence accuracy (%): 93.1 -> 92.6 | No clear effect |
| Transfer to ETHICS in zero-shot mode | Kaleido · ETHICS average test | performance (%): 40.6 -> 43.7 | Helped |
| Fit logistic-regression label calibration to model predictions | Kaleido decision system · CommonsenseNormBank SBIC | accuracy: +4.9 | Helped |
| Fit logistic-regression label calibration to model predictions | Kaleido decision system · CommonsenseNormBank ETHICS | accuracy: +0.1 | No clear effect |
| Fit logistic-regression label calibration to model predictions | Kaleido decision system · CommonsenseNormBank Moral Stories | accuracy: +0.8 | Helped |
Surprising findings
- The 3B student's system outperformed its GPT-4 teacher on overall preference, coverage, and accuracy in pairwise human evaluation.
- Relevance correlation with human judgments does not improve monotonically with model size.
Limitations
- All crowdworkers were U.S.-based, so value representation across nationalities is not captured.
- Training data is GPT-4-generated, so teacher biases and majority-value bias may carry over.
- Relevance score correlates only weakly with human relevance judgments (0.25-0.31).
Experimental setup
Datasets: ValuePrism ETHICS CommonsenseNormBank MoralChoice SocialChem Delphi situations (user queries)
Benchmarks: ValuePrism test split ETHICS CommonsenseNormBank (SBIC, ETHICS, MoralStories, SocialChem) MoralChoice SocialChem
Metrics: win rate vs GPT-4 (overall, coverage, accuracy) explanation correctness valence correctness correlation with human relevance accuracy F1
Baselines: GPT-4 GPT-3.5-turbo ChatGPT Delphi
Builds on: symbolic knowledge distillation Delphi Flan-T5 ETHICS benchmark
Closest papers by meaning
- Position: A Roadmap to Pluralistic Alignment (ICML 2024 · similarity 0.90)
- Learning Human-like Representations to Enable Learning Human Values (NeurIPS 2024 · similarity 0.88)
- Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs (NeurIPS 2025 · similarity 0.88)
- DailyDilemmas: Revealing Value Preferences of LLMs with Quandaries of Daily Life (ICLR 2025 · similarity 0.88)
- Value Imprint: A Technique for Auditing the Human Values Embedded in RLHF Datasets (NeurIPS 2024 · similarity 0.87)
- Many LLMs Are More Utilitarian Than One (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.