Research map / Large language models / Language, safety and interpretability

DENEVIL: TOWARDS DECIPHERING AND NAVIGATING THE ETHICAL VALUES OF LARGE LANGUAGE MODELS VIA INSTRUCTION LEARNING

ICLR 2024 · Large language models › Language, safety and interpretability · approach: Bias measurement and mitigation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DeNEVIL, a dynamic generative prompt-generation method based on EM and simulated annealing to probe LLM ethical value violations, builds the 2,397-prompt MoralPrompt dataset, benchmarks 27 LLMs, and introduces VILMO, a model-based black-box in-context alignment method that learns to generate value instructions.

Problem

Static discriminative evaluations of LLM ethical values (moral judgement and moral questionnaires) suffer from data contamination and measure knowledge of values rather than behavioural conformity to them, and most LLMs appear misaligned with moral foundations.

Approach

Treat value assessment as generative: iteratively refine prompts (EM with simulated annealing) so that the model's completions violate a given value principle, measure violation with a trained classifier, and align black-box LLMs by training a generator to produce prompt-specific value instructions optimized by model-based black-box optimization.

Key results

• Across 27 LLMs, most models show high value violation; ChatGPT shows the best value conformity among those evaluated, outperforming GPT-4 on its own moral prompts.
• VILMO on ChatGPT achieves the lowest violation metrics (EVR 89.45, MVP 85.84, APV 57.58) versus ChatGPT's 96.18, 93.58, 70.07 and APE/InstructZero/Self-Critique baselines.
• Human evaluation (Krippendorff's alpha 0.82) reports VILMO improves value conformity with comparable generation quality.

Measured effects reported in the paper

Surprising findings

• InstructZero performs worse than the simpler APE baseline, attributed to few learnable parameters and little training data.
• Discriminative moral judgement yields the highest value scores, while generative MoralPrompt evaluation yields the lowest, suggesting models know values but do not behave accordingly.

Limitations

• Relies on Moral Foundations Theory only and a single value-principle source (Moral Stories), limiting coverage.
• Classifier used to measure violation has only moderate OOD accuracy (~80%), and evaluation is English-only.
• Potential for misuse to generate harmful content at scale against deployed LLMs.

Experimental setup

Datasets: Moral Stories Social Chemistry MoralPrompt

Benchmarks: MoralPrompt Moral Stories Moral Foundations Questionnaire (MFQ-30)

Metrics: EVR APV MVP Self-BLEU PPL human rating of value conformity and quality

Baselines: ChatGPT Self-Critique APE InstructZero

Builds on: Moral Foundations Theory Moral Stories Self-Critique APE InstructZero

Closest papers by meaning

• Unveiling the Bias Impact on Symmetric Moral Consistency of Large Language Models (NeurIPS 2024 · similarity 0.91)
• CLAVE: An Adaptive Framework for Evaluating Values of LLM Generated Responses (NeurIPS 2024 · similarity 0.90)
• Raising the Bar: Investigating the Values of Large Language Models via Generative Evolving Testing (ICML 2025 · similarity 0.90)
• HonestLLM: Toward an Honest and Helpful Large Language Model (NeurIPS 2024 · similarity 0.89)
• Tracing Moral Foundations in Large Language Models (NeurIPS 2026 · similarity 0.88)
• Language Model Alignment in Multilingual Trolley Problems (ICLR 2025 · similarity 0.88)