AI Paper Atlas

Research map / Large language models / AI and society

HearSayBench: Can LLMs Navigate from Abstract Human Rights to Lived Lives?

Sobhan Lotfi, Ava Iranmanesh, Ali Iranmanesh, Liwei Jiang

NeurIPS 2026 · Large language models › AI and society · approach: Critiques of LLM claims and judging

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces HearSayBench, a benchmark probing whether LLMs can translate abstract human rights into advice suited to users' lived constraints such as war or statelessness.

Problem

LLM advisors on life-critical problems implicitly assume users have full personal agency (the internet reflects people free to act), so existing benchmarks miss users in war zones or stateless situations.

Approach

A benchmark framing that tests whether LLMs can bridge abstract human-rights principles and the concrete constraints of people with limited agency, rather than assuming the freedom to act.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use a stronger model within a family (Claude-Sonnet-4.6 to Claude-Opus-4.8)Claude models · HearsayBenchCapability: 2.92 -> 3.63Improved
Use a stronger model within a family (Claude-Sonnet-4.6 to Claude-Opus-4.8)Claude models · HearsayBenchHarm average: 2.95 -> 2.86Hurt
Use a stronger Gemini model (Gemini-3-Flash to Gemini-3.1-Pro)Gemini models · HearsayBenchHarm average: 2.80 -> 2.91No clear effect
Use a stronger Gemini model (Gemini-3-Flash to Gemini-3.5-Flash)Gemini models · HearsayBenchHarm average: 2.80 -> 2.85No clear effect
Use a stronger Gemini model (Gemini-3-Flash to Gemini-3.1-Pro)Gemini models · HearsayBenchCapability: 3.02 -> 3.63Improved
Use open-weight instead of proprietary modelsLLM advisors · HearsayBenchComprehension-Capability gap: 19% vs 30%Hurt
Use stronger models (top three vs weakest four)LLM advisors · HearsayBenchComprehension-Capability gap: 12 to 13% vs 28 to 37%Improved
Use stronger models (higher Weighted Overall)LLM advisors · HearsayBenchTone minus Harm average gap: r = 0.96Hurt
Score Capability instead of Comprehension for the same modelAll 14 models · HearsayBenchscore (1-5): 0.85 points average dropHurt
Hand-written prompts instead of LLM-generated prompts (ablation)Benchmark construction · HearsayBench scenariosprompt quality (qualitative)Improved
Condition scenarios on Power Relations vs other subtypesLLM advisors · HearsayBenchHarm average: 2.05 (physical harm)Hurt

Experimental setup

Benchmarks: HearSayBench

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy