Research map / Large language models / Language, safety and interpretability
Distributional Open-Ended Evaluation of LLM Cultural Value Alignment Based on Value Codebook
Jaehyeok Lee, Xiaoyuan Yi, Jing Yao, Hyunjin Hwang, Roy Ka-Wei Lee, Xing Xie, JinYeong Bak
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add value codebook (ablation) | DOVE · Predictive validity | Predictive validity: 5.49% -> 31.56% | Improved |
| add codebook refinement (ablation) | DOVE · Predictive validity | Predictive validity: 8.98% -> 31.56% | Improved |
| add UOT metric instead of cosine similarity (ablation) | DOVE · Predictive validity | Predictive validity: 13.16% -> 31.56% | Improved |
| add redundancy reduction (ablation) | DOVE · Predictive validity | Predictive validity: 21.54% -> 31.56% | Improved |
| use GPT-5.2 as value recognizer instead of GPT-5 nano (ablation) | DOVE · Predictive validity | Predictive validity: 28.11% -> 31.56% | Improved |
| use GPT-5.2 as value recognizer instead of gpt-oss-120b (ablation) | DOVE · Predictive validity | Predictive validity: 28.62% -> 31.56% | Improved |
| replace Mauve on document embeddings with Mauve on value code vectors | Mauve · Predictive validity | Predictive validity: -5.73% -> 29.78% | Improved |
| use value expression embeddings rather than document embeddings for Mauve | Mauve · Predictive validity | Predictive validity: -5.73% -> 23.04% | Improved |
| use UOT metric instead of Mauve on value code vectors | DOVE · Predictive validity | Predictive validity: 29.78% -> 31.56% | Improved |
| use DOVE open-ended evaluation instead of WVS | evaluation method · Value priming | Value priming Δg: 0.08% -> 5.60% | Improved |
| use DOVE instead of NormAd | evaluation method · Predictive validity | Predictive validity: 0.90% -> 31.56% | Improved |
| use DOVE instead of NaVAB for discriminant validity | evaluation method · Discriminant validity | Discriminant validity δdis: -88.00% -> 0.89% | Hurt |
Closest papers by meaning
- CLAVE: An Adaptive Framework for Evaluating Values of LLM Generated Responses (NeurIPS 2024 · similarity 0.88)
- CultureLLM: Incorporating Cultural Differences into Large Language Models (NeurIPS 2024 · similarity 0.88)
- On Evaluating LLM Alignment by Evaluating LLMs as Judges (NeurIPS 2025 · similarity 0.87)
- Measuring Human and AI Values Based on Generative Psychometrics with Large Language Models (AAAI 2025 · similarity 0.87)
- ALI-Agent: Assessing LLMs' Alignment with Human Values via Agent-based Evaluation (NeurIPS 2024 · similarity 0.87)
- LLM Alignment--Utility Asymmetry under Semantic-Preserving Transformations (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
