Research map / Large language models / Language, safety and interpretability
A Language Model’s Guide Through Latent Space
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends concept guidance (linear probing plus activation steering) from truthfulness to appropriateness, humor, creativity and quality on Llama-2-7b-chat and Mistral-7B, and proposes a perplexity-normalized effect size metric.
Problem
Concept guidance via linear probes and activation steering has mostly been studied for truthfulness; it is unclear whether detection and guidance transfer to richer concepts like appropriateness, humor, creativity and quality, and how to measure guidance quality jointly with fluency.
Approach
Compare logistic regression, difference-in-means and PCA probes on normalized residual activations of the first t tokens, steer by adding the probe direction with norm preservation across the top-k layers, and score guidance with a perplexity-normalized effect size (PNES) that fits tanh effect and log-quadratic perplexity curves.
Key results
- Logistic regression probes reach roughly 90% detection accuracy in late layers of Llama-2-chat and 85% in Mistral-instruct for appropriateness.
- Compliance is controllable via the appropriateness direction, with PNES 1.00 for Mistral base at k=16 (PCA), indicating concept confusion rather than appropriateness-specific control.
- Only truthfulness clearly benefits from higher detection accuracy for guidance; better detectors do not reliably make better guides.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Guide along the truthfulness direction | LLMs using linear concept guidance · Truthful | PNES | Helped |
| Guide with the appropriateness vector | LLMs using linear concept guidance · Compliance | PNES: 1.00 | Helped |
| Use the appropriateness vector to elicit appropriateness | LLMs using linear concept guidance · Appropriate | appropriateness elicitation | Hurt |
| Use the humor concept vector with a tuned guidance configuration | LLMs using linear concept guidance · Humor | PNES: 0.35 | Helped |
| Use the creativity concept vector with a tuned guidance configuration | LLMs using linear concept guidance · Creativity | PNES: 0.23 | Helped |
| Use the quality concept vector with a tuned guidance configuration | LLMs using linear concept guidance · Quality | PNES: 0.54 | Helped |
| Increase context length from a single token to t=16 (ablation) | Concept probes · Concept detection | accuracy | Mixed |
| Increase context length for PCA probing (ablation) | PCA concept probes · Concept detection | accuracy | Hurt |
| Use logistic regression instead of the other probing techniques (ablation) | Concept detection · Appropriateness/harmfulness | accuracy: approximately 90% | Helped |
| Fine-tune the base models with instruction tuning/alignment (ablation) | Llama-2 and Mistral · Truthfulness, appropriateness, and quality detection | accuracy | Helped |
| Fine-tune the base models with instruction tuning/alignment (ablation) | Llama-2 and Mistral · Humor and creativity detection | accuracy | No clear effect |
| Select probes based on higher detection accuracy (ablation) | Concept guidance configurations · Concept guidance | PNES | No clear effect |
Surprising findings
- Steering along the appropriateness direction controls compliance independent of user intent rather than improving appropriateness.
- Higher probing accuracy does not predict better guidance for most concepts except truthfulness.
Limitations
- Guidance success requires extensive manual tuning of layer count and strength; no configuration works consistently across concepts or models.
- Evaluation relies on few-shot Mistral-7B LLM judges and a small sample of 64 prompts per configuration.
- Appropriateness dataset is synthetic and labels reflect the generator's compliance or refusal rather than ground truth appropriateness.
Experimental setup
Datasets: OpenAssistant (OASST1) ToxicChat TruthfulQA
Benchmarks: TruthfulQA OpenAssistant concept labels (humor, creativity, quality) ToxicChat-based appropriateness/compliance
Metrics: detection accuracy perplexity-normalized effect size (PNES) perplexity concept presence probability
Baselines: Logistic regression probe Difference-in-means probe PCA probe
Builds on: linear representation hypothesis classifier-guided generation inference-time intervention
Closest papers by meaning
- Latent Principle Discovery for Language Model Self-Improvement (NeurIPS 2025 · similarity 0.87)
- Monitoring Latent World States in Language Models with Propositional Probes (ICLR 2025 · similarity 0.87)
- Evaluating the Zero-shot Robustness of Instruction-tuned Language Models (ICLR 2024 · similarity 0.86)
- Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting (ICLR 2024 · similarity 0.86)
- Learning Robust Rationales for Model Explainability: A Guidance-Based Approach (AAAI 2024 · similarity 0.86)
- When Is Rank-1 Steering Cheap? Geometry, Granularity, and Budgeted Search (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.