AI Paper Atlas

Research map / Large language models / AI and society

Evaluating Online Moderation via LLM-Powered Counterfactual Simulations

Giacomo Fidone, Lucia Passaro, Riccardo Guidotti

AAAI 2026 · Large language models › AI and society · approach: Critiques of LLM claims and judging · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use personalized neutral moderation message (PMI-N) instead of one-size-fits-all message (ablation)COSMOS ex ante moderation · COSMOS full-population simulationsmass divergence of toxicity: OSFA -0.06, +0.04, -0.05, -0.06, +0.05 vs PMI-N -0.09, +0.00, -0.08, -0.11, -0.11Improved
make personalized moderation prescriptive (PMI-P) instead of neutral (PMI-N) (ablation)COSMOS PMI · COSMOS full-population simulation 5mass divergence of toxicity: -0.11 -> -0.02Hurt
make personalized moderation empathizing (PMI-E) instead of neutral (PMI-N) (ablation)COSMOS PMI · COSMOS full-population simulation 4mass divergence of toxicity: -0.11 -> -0.02Hurt
lower ban tolerance from e=8 to e=1 (ablation)COSMOS BAN · COSMOS full-population simulation 1mass divergence of toxicity: -0.06 -> -0.54Improved
lower ban tolerance from e=8 to e=1 (ablation)COSMOS BAN · COSMOS full-population simulationscontent loss ratio: CLR 0.04 -> 0.45Hurt
lower ban tolerance to e=1 (healthy content lost) (ablation)COSMOS BAN · COSMOS full-population simulationsfraction of healthy content lost: 0.27 ± 0.03 of content below THR lost; macro-average recall 0.55 at e=1, 0.60 at e=2, 0.58 at e=4Hurt
use cal_tox prompt template instead of no_tox (ablation)COSMOS user prompt · PANDORA toxicity distributionKL divergence: 1.37 -> 0.07Improved
use cal_tox prompt template instead of yes_tox (ablation)COSMOS user prompt · PANDORA toxicity distributionKL divergence: 0.57 -> 0.07Improved
use cal_tox instead of no_tox template (ablation)COSMOS user prompt · candidate template evaluationpercent toxic outputs: 0.03 -> 8.12Improved
use cal_tox instead of no_tox template (ablation)COSMOS user prompt · candidate template evaluationpercent correctly formatted outputs: 84.95 -> 71.66Hurt
use SOLAR-10B instead of Llama-2-13B (ablation)COSMOS LLM · PANDORA sampleperplexity: 53.63 -> 22.50Improved
restrict simulation to sub-population of 5 most toxic agents plus least toxic agentCOSMOS · sub-population simulationagent median toxicity: parent-child toxicity correlation rho=+0.39Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy