Research map / Large language models / Agents, code and math
CLASH: Evaluating Language Models on Judging High-Stakes Dilemmas from Multiple Perspectives
Ayoung Lee, Ryan Sungmo Kwon, Peter Railton, Lu Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use thinking (reasoning) models instead of non-thinking counterparts | LLMs on CLASH · CLASH overall | accuracy: Claude-4-Sonnet 88.89 vs Claude-3.5-Sonnet 85.37 | Improved |
| use thinking models on ambivalent (Simple Contrast) cases | Qwen, GPT families · CLASH Simple Contrast | accuracy: GPT-4o 38.37 -> GPT-5 24.06 | Hurt |
| ask about current instead of previous values after a value shift | all models · CLASH Half-Shift/False-Shift | accuracy: average drop 42.95 points, max 66.43 (GPT-4o-mini) | Hurt |
| use more backward chaining and verification in reasoning | thinking models · CLASH reasoning chain analysis | success vs failure of chain | Hurt |
| early commitment and overcommitment in reasoning chains | thinking models · CLASH reasoning chain analysis | success vs failure of chain | Hurt |
| reason with pragmatic and rights-based ethics | thinking models · CLASH reasoning chain analysis | success of chain | Improved |
| stronger base preference for a value | LLM steerability · CLASH value pairs | conditional steerability toward opposing value (Spearman r): r = -0.243, p < 0.005 | Hurt |
| use larger models (vs smaller) for steerability | LLMs · CLASH conditional steerability | aggregate steerability: p<0.0001 | Improved |
| switch from third-person to first-person question framing (ablation) | LLMs · Straightforward, Simple, Swayed Contrast | accuracy and steerability: Wilcoxon p < 0.001 | Hurt |
| first-person framing when steering toward Safety vs Self-Esteem (ablation) | LLMs · Safety vs Self-Esteem pair | steerability | Improved |
| summarize the detailed situation narratives (ablation) | CLASH situations · six open models | accuracy (original minus summarized): overall General 0.65*, Discomfort 1.25*, Temporal -0.69* | Mixed |
Closest papers by meaning
- LLM Strategic Reasoning: Agentic Study through Behavioral Game Theory (NeurIPS 2025 · similarity 0.90)
- LLM Agents Can Be Choice-Supportive Biased Evaluators: An Empirical Study (AAAI 2025 · similarity 0.89)
- EAI: Emotional Decision-Making of LLMs in Strategic Games and Ethical Dilemmas (NeurIPS 2024 · similarity 0.88)
- GTBench: Uncovering the Strategic Reasoning Capabilities of LLMs via Game-Theoretic Evaluations (NeurIPS 2024 · similarity 0.88)
- Many LLMs Are More Utilitarian Than One (NeurIPS 2025 · similarity 0.87)
- Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
