AI Paper Atlas

Research map / Large language models / AI and society

PluriHarms: Benchmarking the Full Spectrum of Human Judgments on AI Harm

Jing-Jing Li, Joel Mire, Eve Fleisig, Valentina Pyatkin, Anne Collins, Maarten Sap, Sydney Levine

ICLR 2026 · Large language models › AI and society · approach: Data commons and human baselines · 6 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
personalize k-shot steering to individual annotators instead of aggregated ratingsGPT-4.1 k-shot · PluriHarmsMAE: 0.254 -> 0.196Improved
use k-shot examples instead of value profile summaries for individual steeringGPT-4.1 · PluriHarms (individual)MAE: 0.233 -> 0.196Improved
steer GPT-4.1 to aggregated ratings with value profile instead of zero-shotGPT-4.1 · PluriHarms (aggregated)MAE: 0.263 -> 0.260No clear effect
align SafetyAnalyst to aggregated ratings rather than WildGuard probability zero-shotSafetyAnalyst · PluriHarms (aggregated)MAE: 0.364 -> 0.361No clear effect
use WildGuard probabilities instead of binary classification labelsWildGuard-7B · PluriHarms (aggregated)MAE: 0.403 -> 0.364Improved
use general model (GPT-4.1 zero-shot) instead of specialized safety model SafetyAnalystsafety models · PluriHarms (aggregated)MAE: 0.361 -> 0.263Improved
personalize SafetyAnalyst to individual annotatorsSafetyAnalyst-8B · PluriHarmsMAE: 0.361 -> 0.311Improved
add prompt-level features to the null model (ablation)mixed-effects regression · PluriHarmsBICImproved
add annotator traits and trait interactions on top of prompt features (ablation)mixed-effects regression · PluriHarmsBICImproved
add annotator traits to explain ratings (ablation)mixed-effects model (random intercept per prompt) · PluriHarmsR2 (fixed effects): 0.0232 -> 0.0686Improved
increase synthetic harm level of promptsDeepSeek-V3 generated prompts · PluriHarmsSpearman correlation with human rating: r = 0.59Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy