Research map / Large language models / Agents, code and math
Where Norms and References Collide: Evaluating LLMs on Normative Reasoning
Mitchell Abrams, Kaveh Eskandari Miandoab, Felix Gervits, Vasanth Sarathy, Matthias Scheutz
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the list of social norms to the prompt (ablation) | 11 LLMs on SNIC · SNIC (9,000) | accuracy: GPT-4o-mini 51.41 -> 93.57; GPT-4.1 59.62 -> 99.6 | Improved |
| add Prolog/FOL formalization to the scene description (ablation) | 11 LLMs on SNIC · SNIC (9,000) | accuracy: average 44.22 -> 44.08 | No clear effect |
| add FOL formalization (ablation) | Llama-3 3B · SNIC (9,000) | accuracy: 33.71 -> 12.01 | Hurt |
| add FOL formalization (ablation) | Granite-3 2B · SNIC (9,000) | accuracy: 27.52 -> 46.65 | Improved |
| add norms to prompt (ablation) | Llama-3 3B · SNIC (9,000) | accuracy: 12.01 -> 26.6 | No clear effect |
Closest papers by meaning
- Beyond Right and Wrong: Evaluating Second-order Social Reasoning in Large Language Models (NeurIPS 2026 · similarity 0.86)
- Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making (NeurIPS 2024 · similarity 0.85)
- Infer Human’s Intentions Before Following Natural Language Instructions (AAAI 2025 · similarity 0.85)
- MetaMind: Modeling Human Social Thoughts with Metacognitive Multi-Agent Systems (NeurIPS 2025 · similarity 0.84)
- Position: LLM Social Simulations Are a Promising Research Method (ICML 2025 · similarity 0.84)
- Theory of Mind and Persuasion Beyond Conversation: Assessing the Capacity of LLMs to Induce Belief States via Planning and Action (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
