Research map / Large language models / Language, safety and interpretability

Value Entanglement: Conflation Between Different Kinds of Good In (Some) Large Language Models

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: Linguistic encoding in LLM representations · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that many open-weight LLMs conflate moral with grammatical and economic value ("value entanglement") in behavioral ratings and residual-stream geometry, and that ablating the morality direction partially repairs the conflation.

Problem

Whether LLMs distinguish different kinds of value (moral, grammatical, economic) as humans do, or conflate them in behavior and internal representations, which matters for value alignment.

Approach

Builds orthogonal stimulus sets (MoralGrammar68, MoralEconomic68) and difference-of-means attribute vectors from residual streams, measuring entanglement via behavioral Likert correlations and activation projections, then uses directional (double) ablation of the morality vector to test causal influence and repair.

Key results

• Open-weight models such as Qwen2.5 7B (r=.46) and Gemma 3 27B (r=.40) show significant correlation between morality and grammaticality ratings, versus human r=.05, while closed models largely do not.
• Human moral ratings are correlated r=.97-.98 with model morality ratings, but model grammaticality ratings correlate with human morality ratings at r=.43 (Qwen2.5 7B).
• Ablating the morality vector raised correlation of Qwen2.5 7B economic ratings with ground truth from r=.20 to r=.55 at peak, repairing part of the entanglement.

Surprising findings

• Ablating the morality vector improved grammaticality and economic ratings, suggesting moral information interferes with those judgments.
• Entanglement was equally present in base (pre-trained only) and instruction-tuned models, suggesting RLHF is not necessary for it.

Limitations

• Entanglement was largely absent in large closed-weight models, so generality of findings across frontier models is limited.
• Economic vector validation correlations are modest for Mistral-Small 24B (r=.49), and ablation effects are variable across layers.
• Stimuli are small (68 sentences) and constructed templates, limiting generality.

Experimental setup

Datasets: MoralGrammar68 MoralEconomic68 Dillion Moral Norms Grand Semantic Controls

Benchmarks: MoralGrammar68 MoralEconomic68 Dillion Moral Norms Grand Semantic Controls

Metrics: Pearson correlation ANOVA difference of correlations

Baselines: Gemma-2 9B Mistral-Small 24B Dillion Moral Norms human ratings human raters

Builds on: difference of means attribute vectors directional ablation emergent misalignment

Closest papers by meaning

• Tracing Moral Foundations in Large Language Models (NeurIPS 2026 · similarity 0.88)
• Distributive Fairness in Large Language Models: Evaluating Alignment with Human Values (NeurIPS 2025 · similarity 0.86)
• DENEVIL: TOWARDS DECIPHERING AND NAVIGATING THE ETHICAL VALUES OF LARGE LANGUAGE MODELS VIA INSTRUCTION LEARNING (ICLR 2024 · similarity 0.86)
• Deep Value Benchmark: Measuring Whether Models Generalize Deep values or Shallow Preferences (NeurIPS 2025 · similarity 0.86)
• Strong Preferences Affect the Robustness of Preference Models and Value Alignment (ICLR 2025 · similarity 0.86)
• Counterfactual Reasoning for Steerable Pluralistic Value Alignment of Large Language Models (NeurIPS 2025 · similarity 0.86)