Research map / Large language models / Alignment and preferences
EigenBench: A Comparative Behavioral Measure of Value Alignment
Jonathn Chang, Leonhard Piff, Suvadip Sana, Jasmine Xinze Li, Lionel Levine
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the reflection step to the judge scaffold (ablation) | EigenBench judges · Judge quality (Claude 3 Haiku) | cycle rate: 0.11 -> 0.06 | Improved |
| add the reflection step to the judge scaffold (ablation) | EigenBench judges · Judge quality (GPT 4.1 Nano) | cycle rate: 0.15 -> 0.05 | Improved |
| add the reflection step to the judge scaffold (ablation) | EigenBench judges · Judge quality (GPT 4.1 Nano) | primacy bias: 0.42 -> 0.24 | Improved |
| add the reflection step to the judge scaffold (ablation) | EigenBench judges · Judge quality (Claude 3.5 Haiku) | recency bias: 0.07 -> 0.10 | Hurt |
| add the reflection step to the judge scaffold (ablation) | EigenBench judges · Judge quality (GPT 4o Mini) | primacy bias: 0.09 -> 0.13 | No clear effect |
| fine-tune Llama 3.1 8b on the Loving constitution (character training) | Llama 3.1 8b · Loving constitution | EigenBench Elo: 1426 -> 1573 | Improved |
| pre-prompt Llama 3.1 8b with the Loving constitution | Llama 3.1 8b · Loving constitution | EigenBench Elo: 1426 -> 1579 | Improved |
| change scenario source from r/AskReddit to OASST (ablation) | EigenBench scores · Universal Kindness | Elo (Grok 4): 1468 -> 1559 | Mixed |
| change scenario source from r/AskReddit to OASST (ablation) | EigenBench scores · Universal Kindness | Elo (GPT 4.1): 1478 -> 1403 | Hurt |
| add Claude 4 Sonnet and Claude 3.5 Haiku to the model population (ablation) | EigenBench scores · Universal Kindness | Elo (Grok 4): 1501 -> 1478 | Hurt |
| replace default prompting by persona pre-prompts across 5 LMs | EigenBench scores · Prompted dispositions (25 models) | variance explained: 79% persona, 21% LM | Mixed |
| compare EigenBench ranking with GPQA ground-truth ranking without labels | EigenBench · GPQA (15 models) | Kendall tau: tau 0.77, 12 adjacent swaps | Improved |
Closest papers by meaning
- Deep Value Benchmark: Measuring Whether Models Generalize Deep values or Shallow Preferences (NeurIPS 2025 · similarity 0.90)
- Learning Human-like Representations to Enable Learning Human Values (NeurIPS 2024 · similarity 0.89)
- Measuring Human and AI Values Based on Generative Psychometrics with Large Language Models (AAAI 2025 · similarity 0.88)
- SEAL: Systematic Error Analysis for Value ALignment (AAAI 2025 · similarity 0.88)
- Strong Preferences Affect the Robustness of Preference Models and Value Alignment (ICLR 2025 · similarity 0.88)
- Measuring Error Alignment for Decision-Making Systems (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
