AI Paper Atlas

Research map / Large language models / AI and society

ELEPHANT: Measuring and understanding social sycophancy in LLMs

Myra Cheng, Sunny Yu, Cinoo Lee, Pranav Khadpe, Lujain Ibrahim, Dan Jurafsky

ICLR 2026 · Large language models › AI and society · approach: Data commons and human baselines

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Compare LLM responses to human responses on open-ended advice11 LLMs · OEQValidation rate: 22% (human) -> 72% (LLM)Hurt
Compare LLM responses to human responses on indirectness11 LLMs · OEQIndirectness rate: 21% (human) -> 66% (LLM)Hurt
Present both sides of a moral conflict to the model11 LLMs · AITA-NTA-FLIPMoral sycophancy: 48% of cases say NTA to both sidesHurt
Use preferred rather than dispreferred responses in preference datasetsLMSys, UltraFeedback, PRISM, HH-RLHF · Preference datasetsValidation and indirectness rateHurt
Preferred vs dispreferred responses on framing sycophancyPreference datasets · Preference datasetsFraming rateNo clear effect
Prepend instruction to be less sycophantic when appropriate (ablation)GPT-4o, Llama-70B · OEQSocial sycophancy score: GPT-4o OEQ validation 0.71, framing -0.58Mixed
Rewrite prompts in the third person (ablation)GPT-4o, Llama-8B, Llama-70B · OEQ, AITA, SSSocial sycophancy score: OEQ validation GPT-4o 0.71 -> 0.45 (instruction vs perspective row)Mixed
Use truthfulness-tuned ITI model (ablation)Llama-70B vs Llama-8B · OEQValidation sycophancy: 8B 0.56; 70B 0.18Mixed
Fine-tune with DPO against validation and indirectness (ablation)Llama-8B · OEQValidation sycophancy: -0.12 (DPO-Validation)Improved
Fine-tune with DPO against framing sycophancy (ablation)Llama-8B · OEQFraming sycophancyNo clear effect
Increase model size within Llama or Mistral familiesLlama and Mistral models · ELEPHANTSocial sycophancyNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy