Research map / Large language models / Language, safety and interpretability

Uncovering Gaps in How Humans and LLMs Interpret Subjective Language

ICLR 2025 · Large language models › Language, safety and interpretability · approach: Faithfulness of LLM explanations · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TED, which builds gradient-based operational thesauruses of subjective phrases for LLMs and flags clashes with human or LLM-constructed semantic thesauruses to uncover misalignments that predict downstream failures in output editing and inference steering.

Problem

LLMs' operational semantics of subjective natural-language phrases (e.g., "enthusiastic", "helpful") may not match human expectations, causing unexpected side effects or inadequate updates when used in prompts or constitutional-style output editing.

Approach

Encode each subjective phrase as the averaged gradient of the LLM's output log-likelihood with respect to the embedding of a generic prompt, compare phrases by cosine similarity to form an LLM operational thesaurus, and search for disagreements with a human- or LLM-built semantic thesaurus; disagreements are then tested downstream with a GPT-4 judge.

Key results

• For Llama 3 8B with the LLM-constructed thesaurus, 23% of output-editing unexpected side-effect pairs reach at least 90% success rate versus 0% for the semantic-only baseline.
• On inference steering with Llama 3 8B and the human thesaurus, over 56% of unexpected side-effect pairs reach at least 90% success versus 10% for the baseline.
• TED outperforms the semantic-only baseline on average success rate across all models, tasks, and thesauruses; e.g., Mistral output-editing unexpected side effects (LLM thesaurus) 75.5 vs 51.1.

Measured effects reported in the paper

Surprising findings

• Mistral 7B Instruct produces more harassing outputs when editing text to be witty (78%), and Llama 3 8B produces more dishonest articles when asked to be enthusiastic (97% in inference steering).
• GPT-4 judgments agreed with the majority human annotation at nearly the level of individual annotators, and agreed 97% of the time when all annotators agreed.

Limitations

• Inadequate-update failures are found less often and TED has lower success rates there, possibly due to false positives.
• Semantic thesauruses are noisy: human labels are costly and LLM-built thesauruses are imperfect proxies for human judgment.
• Success is judged by GPT-4 rather than humans on the full set, and ties reduce measured success.

Experimental setup

Datasets: GPT-4-generated ethical questions (100 for embeddings, 100 for testing) GPT-4-generated writing topics for inference steering human-labeled pairs from Amazon Mechanical Turk (1260 pairs) LLM-constructed semantic thesaurus from GPT-4

Benchmarks: output-editing failures on ethical questions inference-steering failures on writing prompts

Metrics: average downstream success rate fraction of pairs with success rate above thresholds 0.1, 0.

Baselines: Semantic-only baseline

Builds on: Constitutional AI function vectors conditional DPO

Closest papers by meaning

• Bridging Human and LLM Judgments: Understanding and Narrowing the Gap (NeurIPS 2025 · similarity 0.88)
• Understanding Annotator Perception: Modeling Psychological Inference from First- and Third-Person Annotations (Student Abstract) (AAAI 2025 · similarity 0.87)
• LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations (ICLR 2025 · similarity 0.87)
• The Grounding Gap: How LLMs Anchor the Meaning of Abstract Concepts Differently from Humans (NeurIPS 2026 · similarity 0.87)
• Do Large Language Models Perform the Way People Expect? Measuring the Human Generalization Function (ICML 2024 · similarity 0.86)
• Walk the Talk? Measuring the Faithfulness of Large Language Model Explanations (ICLR 2025 · similarity 0.86)