Research map / Large language models / Language, safety and interpretability
Controlling Large Language Models Through Concept Activation Vectors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GCAV, a lightweight activation-engineering framework that trains logistic-regression concept activation vectors from contrastive prompts and steers LLM activations with per-sample closed-form strengths to control toxicity, sentiment, topic and style without fine-tuning.
Problem
Controlled LLM generation is costly (fine-tuning) or coarse (prompting, decoding interventions), and existing activation steering uses fixed steering strengths that do not adapt to each input.
Approach
Train a concept activation vector (CAV) per concept from contrastive prompts via a logistic-regression classifier on layer activations, then add the vector to chosen layers with a closed-form, per-sample steering strength that just meets a target concept probability, including joint multi-concept steering via a constrained optimization.
Key results
- On Llama-2-7b-chat toxicity_toxic, GCAV-Output reduces toxicity to 0.0879 versus 0.1807 for BASE and 0.1620 for ActAdd.
- On sentiment control with Llama-2-7b-chat, GCAV-Output achieves 0.5566 success versus 0.5280 for POSPROMPT and 0.4240 for ActAdd.
- Steering strength for multi-concept control increases with topic strength in GCAV, while ActAdd fails to keep other concepts stable.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GCAV-Output instead of Arithmetic | toxicity reduction on Llama-2-7b-chat · toxicity_toxic | toxicity: 0.1625 -> 0.0879 | Helped |
| Use GCAV-Output instead of Arithmetic | toxicity reduction on Llama-2-7b-chat · toxicity_random | toxicity: 0.0816 -> 0.0622 | Helped |
| Train CAVs from filtered outputs rather than concept-related input prompts (ablation) | GCAV toxicity reduction on Llama-2-7b-chat · toxicity_toxic | toxicity: 0.1231 -> 0.0879 | Helped |
| Train CAVs from filtered outputs rather than concept-related input prompts (ablation) | GCAV toxicity reduction on Llama-2-7b-chat · toxicity_random | toxicity: 0.0666 -> 0.0622 | Helped |
| Train CAVs from filtered outputs rather than concept-related input prompts (ablation) | GCAV toxicity reduction on Llama-2-7b · toxicity_toxic | toxicity: 0.3494 -> 0.3962 | Hurt |
| Train CAVs from filtered outputs rather than concept-related input prompts (ablation) | GCAV toxicity reduction on Llama-2-7b · toxicity_random | toxicity: 0.1005 -> 0.0998 | Helped |
| Use GCAV-Output instead of POSPROMPT | sentiment control on Llama-2-7b-chat · IMDB sentiment control | Success: 0.5280 -> 0.5566 | Helped |
| Use GCAV-Output instead of Arithmetic | sentiment control on Llama-2-7b · IMDB sentiment control | Success: 0.4960 -> 0.4830 | Hurt |
Surprising findings
- Arithmetic decoding yields very short, off-topic responses that get low toxicity but very high perplexity, so it was excluded from the Llama-2-7b comparison.
- Prompting with positive instructions (POSPROMPT) can increase perplexity and fluency cost and sometimes raises toxicity.
Limitations
- Performance depends on the quality of the contrastive prompts used to train CAVs.
- Evaluation is limited to Llama-2 models and a small set of concepts and single-run comparisons.
- Some baselines (Arithmetic) are impractical due to high perplexity, limiting direct comparison.
Experimental setup
Datasets: RealToxicityPrompts IMDB movie reviews GPT-4o-generated contrastive prompts
Benchmarks: RealToxicityPrompts (toxicity_toxic, toxicity_random) IMDB sentiment continuation
Metrics: Toxicity (Perspective API) perplexity fluency control success rate topic strength formality accuracy
Baselines: BASE POSPROMPT Arithmetic ActAdd
Builds on: Concept Activation Vectors (TCAV) ActAdd Arithmetic decoding
Closest papers by meaning
- Controlling Language and Diffusion Models by Transporting Activations (ICLR 2025 · similarity 0.88)
- Enhancing Large Language Model Performance with Gradient-Based Parameter Selection (AAAI 2025 · similarity 0.87)
- Controlled Text Generation via Language Model Arithmetic (ICLR 2024 · similarity 0.86)
- Cross-model Control: Improving Multiple Large Language Models in One-time Training (NeurIPS 2024 · similarity 0.86)
- Dictionary-Aligned Concept Control for Safeguarding Multimodal LLMs (CVPR 2026 · similarity 0.86)
- Steering When Necessary: Flexible Steering Large Language Models with Backtracking (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.