Research map / Large language models / AI and society
Sycophancy Towards Researchers Drives Performative Misalignment
David D. Baek, Xinnuo Li, Anay Gupta, Taslim Mahbub, Kejian Shi, Max Tegmark, Shi Feng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a deployment system prompt instead of an evaluation prompt | LLM evaluation-awareness classification · StyleCues | EVAL probability: 15% to 55% remains under deployment prompt | Mixed |
| Use researcher-style writing instead of ordinary user writing | LLM evaluation-awareness classification · StyleCues | EVAL probability: increases ranging from 13% to 27% | Improved |
| Use thinking variant instead of instruct variant | Qwen3-80B · StyleCues (deployment prompt) | EVAL detection rate (%): 11.7 -> 42.5 (+30.8) | Improved |
| Use thinking variant instead of instruct variant | Qwen3-235B · StyleCues (deployment prompt) | EVAL detection rate (%): 39.6 -> 48.9 (+9.3) | Improved |
| Use thinking variant instead of instruct variant | Kimi-K2 · StyleCues (deployment prompt) | EVAL detection rate (%): 50.0 -> 78.2 (+28.2) | Improved |
| Compare monitored (Free) vs unmonitored (Paid) tier for A3 rating features | GPT-OSS-20B SAE probing · Alignment-faking-rl | mean activation: 0.04 -> 0.07 (+0.03) | Improved |
| Compare monitored vs unmonitored tier for B1 detection-risk features | GPT-OSS-20B SAE probing · Alignment-faking-rl | mean activation: 0.14 -> 0.15 (+0.01) | No clear effect |
| Steer with sycophancy-related SAE features instead of scheming-related features | GPT-OSS-20B · StyleCues | evaluation awareness | No clear effect |
| Fine-tune on sycophancy datasets | GPT-OSS-120B, Qwen-30B, Qwen-235B · StyleCues | evaluation awareness: p=0.016, 0.012, 0.037 | Improved |
| Fine-tune on sycophancy datasets | GPT-OSS-20B · StyleCues | evaluation awareness: p=0.089 | No clear effect |
| Fine-tune on sycophancy datasets | Base models · MMLU, TruthfulQA, StrongREJECT, free-form, deception | misalignment: <5pp increase | No clear effect |
Closest papers by meaning
- When Helpfulness Becomes Sycophancy: Sycophancy is a Boundary Failure Between Social Alignment and Epistemic Integrity in Large Language Models (NeurIPS 2026 · similarity 0.87)
- Why Do Some Language Models Fake Alignment While Others Don't? (NeurIPS 2025 · similarity 0.86)
- Training on the Test Task Confounds Evaluation and Emergence (ICLR 2025 · similarity 0.85)
- DeceptionBench: A Comprehensive Benchmark for AI Deception Behaviors in Real-world Scenarios (NeurIPS 2025 · similarity 0.85)
- Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs (ICML 2025 · similarity 0.85)
- Trust, but Don’t Verify: Epistemic Blind Spots in LLM Source Evaluation (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
