Research map / Large language models / Language, safety and interpretability
AdAEM: An Adaptively and Automated Extensible Evaluation Method of LLMs' Value Difference
Jing Yao, Shitong Duan, Xiaoyuan Yi, Dongkuan Xu, Peng Zhang, Tun Lu, Ning Gu, Zhicheng Dou, Xing Xie
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use AdAEM generated questions instead of human-created general questions | value evaluation questions · human evaluation, 300 questions | reasonableness: 8.7% | Improved |
| use AdAEM generated questions instead of human-created general questions | value evaluation questions · human evaluation, 300 questions | value differentiation: 52% | Improved |
| use AdAEM Bench instead of ValueDCG for question set | question diversity · benchmark statistics | Self-BLEU: 13.93 -> 13.42 | Improved |
| use AdAEM Bench instead of ValueDCG for question set | question novelty · benchmark statistics | semantic similarity: 0.36 -> 0.44 | Hurt |
| use AdAEM Bench instead of SVS for question set | question diversity · benchmark statistics | Self-BLEU: 52.68 -> 13.42 | Improved |
| prime o3-mini toward a target value | AdAEM measurement · value priming | value score change: +31% | Improved |
| prime o3-mini toward a target value, effect on opposing values | AdAEM measurement · value priming | value score change: -58% | Hurt |
| prime o3-mini toward a target value, effect on same-group values | AdAEM measurement · value priming | value score change: +17% | Improved |
| increase optimization budget B | AdAEM · informativeness vs diversity | informativeness score and topic diversity | Mixed |
| use AdAEM questions instead of SVS and ValueDCG for LLM comparison | value evaluation · LLM value comparison | distinguishability of LLM values | Improved |
| use GPT-4o instead of GPT-4o-Mini as value classifier | value judgement · target domain | accuracy: 92.60 -> 92.92 | No clear effect |
| increase LLM size from 8B to 405B | Llama models · AdAEM Bench | Tradition and Universalism preference | Improved |
Closest papers by meaning
- CLAVE: An Adaptive Framework for Evaluating Values of LLM Generated Responses (NeurIPS 2024 · similarity 0.94)
- Raising the Bar: Investigating the Values of Large Language Models via Generative Evolving Testing (ICML 2025 · similarity 0.90)
- Measuring Human and AI Values Based on Generative Psychometrics with Large Language Models (AAAI 2025 · similarity 0.89)
- Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (ICLR 2025 · similarity 0.89)
- LLMEval: A Preliminary Study on How to Evaluate Large Language Models (AAAI 2024 · similarity 0.88)
- Bridging Human and LLM Judgments: Understanding and Narrowing the Gap (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
