Research map / Large language models / Language, safety and interpretability
Evaluating the Moral Beliefs Encoded in LLMs
Nino Scherrer, Claudia Shi, Amir Feder, David Blei
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Evaluate responses across multiple question forms and action orderings | LLMs answering MoralChoice-LowAmbiguity · MoralChoice-LowAmbiguity | response consistency | Mixed |
| Evaluate responses across multiple question forms and action orderings | Some smaller OpenAI models · MoralChoice-LowAmbiguity | question-form consistency | Hurt |
| Use multiple question forms to diagnose uncertainty | Some Google and BigScience models · MoralChoice-LowAmbiguity | action uncertainty | No clear effect |
| Apply human-preference alignment procedures during fine-tuning | Commercial API models · MoralChoice-HighAmbiguity | agreement in marginal action likelihood: all pairwise coefficients ≥ 0.75 | Improved |
Closest papers by meaning
- Unveiling the Bias Impact on Symmetric Moral Consistency of Large Language Models (NeurIPS 2024 · similarity 0.90)
- Decision-Making Behavior Evaluation Framework for LLMs under Uncertain Context (NeurIPS 2024 · similarity 0.88)
- Language Model Alignment in Multilingual Trolley Problems (ICLR 2025 · similarity 0.88)
- DailyDilemmas: Revealing Value Preferences of LLMs with Quandaries of Daily Life (ICLR 2025 · similarity 0.88)
- Many LLMs Are More Utilitarian Than One (NeurIPS 2025 · similarity 0.86)
- DENEVIL: TOWARDS DECIPHERING AND NAVIGATING THE ETHICAL VALUES OF LARGE LANGUAGE MODELS VIA INSTRUCTION LEARNING (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
