Research map / Large language models / Instruction and fine-tuning
PaCE: Parsimonious Concept Engineering for Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PaCE, a training-free activation engineering method that builds a 40k-concept activation dictionary (PaCE-1M), partitions concepts with GPT-4 per task, and removes undesirable components via sparse-coding oblique projection of LLM activations.
Problem
Activation engineering for LLM alignment (detoxification, faithfulness, sentiment) struggles to remove undesirable concepts without removing benign ones, because existing vector addition and orthogonal projection methods model the activation geometry poorly and remove only one direction.
Approach
Model many concept directions in an overcomplete dictionary, decompose activations with elastic-net sparse coding (oblique projection), and zero out only the coefficients of concepts a GPT-4 partitioner labels undesirable, keeping the residual.
Key results
- On SafeEdit with LLaMA2-13B-Chat, PaCE reaches 93.7-99.3% safety across most categories, dominating VecAdd and OrthoProj in all nine categories.
- On LLaMA2-7B-Chat, PaCE improves safety by 60-80% over vanilla in IA, MH, OF, PH, PP, and UB, and by 6-20% over other activation baselines, with fluency 8.07 vs 7.70 for vanilla.
- PaCE takes about 3x less time per response than OrthoProj (44.8s vs 143.7s on LLaMA2-7B-Chat), though it is slower than VecAdd.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PaCE instead of the strongest reported baseline | LLaMA-7B-Chat detoxification · SafeEdit effective set, IA | safety (%): 65.2 -> 75.3 | Helped |
| Use PaCE instead of the strongest reported baseline | LLaMA2-13B-Chat detoxification · SafeEdit effective set, IA | safety (%): 64.3 -> 94.9 | Helped |
| Use PaCE instead of Vanilla | LLaMA-7B-Chat detoxification · SafeEdit effective set, MMLU | accuracy (%): 43.4 -> 37.1 | Hurt |
| Use PaCE instead of Vanilla | LLaMA2-13B-Chat detoxification · SafeEdit effective set, MMLU | accuracy (%): 54.9 -> 54.1 | Hurt |
| Use PaCE instead of the strongest reported baseline | LLaMA2-7B-Chat · FactScore labeled subset | LAF: 49.7 -> 65.9 | Helped |
| Use PaCE instead of the strongest reported baseline | LLaMA2-7B-Chat · HolisticBias, Gender | non-negative sentiment (%): 55.2 -> 66.2 | Helped |
| Use PaCE instead of the strongest reported baseline | LLaMA2-13B-Chat · HolisticBias, Gender | non-negative sentiment (%): 57.7 -> 63.4 | Helped |
| Add decomposition on the top 10,000 concepts (ablation) | PaCE detoxification · Detoxification ablation | safety (%): 50.2 -> 57.6 | Helped |
| Add concept clustering (ablation) | PaCE detoxification with 10,000 concepts · Detoxification ablation | safety (%): 57.6 -> 62.3 | Helped |
| Add the GPT-4 concept partitioner (ablation) | PaCE detoxification with clustering · Detoxification ablation | safety (%): 62.3 -> 65.1 | Helped |
| Remove the top 50 undesirable concepts (ablation) | PaCE detoxification with concept partitioning · Detoxification ablation | safety (%): 65.1 -> 76.5 | Helped |
Surprising findings
- Prompting-based instruction obtains the highest PS safety on LLaMA2-7B but degrades MMLU accuracy to 15.4% from 43.4% vanilla.
- Applying OrthoProj or VecAdd across many undesirable concepts without decomposition degrades linguistic capability, which motivates the sparse decomposition.
Limitations
- Computationally slower than VecAdd, with per-response time roughly 3.6x vanilla on LLaMA2-7B.
- Concept partitioning depends on GPT-4 annotations, which may be contentious or culturally misaligned.
- Concept representation uses a single vector per concept, which may not handle polysemy well; results evaluated only on LLaMA2 chat models.
Experimental setup
Datasets: Brown Corpus Wikipedia (FAISS-indexed 2018 snapshot) SafeEdit FactScore HolisticBias MMLU AdvBench
Benchmarks: SafeEdit MMLU AdvBench FactScore HolisticBias
Metrics: safety score (%) fluency perplexity MMLU accuracy (%) factuality (labeled/unlabeled score) sentiment score (%) computation time
Baselines: Prompting VecAdd OrthoProj PE SFT RLHF
Builds on: representation engineering vector addition (activation addition) orthogonal projection (debiasing) elastic net representation reading
Closest papers by meaning
- Aligner: Efficient Alignment by Learning to Correct (NeurIPS 2024 · similarity 0.89)
- LLM Alignment--Utility Asymmetry under Semantic-Preserving Transformations (NeurIPS 2026 · similarity 0.89)
- Exploring Intrinsic Alignments Within Text Corpus (AAAI 2025 · similarity 0.89)
- Differentially Private Steering for Large Language Model Alignment (ICLR 2025 · similarity 0.88)
- Peering Through Preferences: Unraveling Feedback Acquisition for Aligning Large Language Models (ICLR 2024 · similarity 0.88)
- Aligning Large Language Models with Representation Editing: A Control Perspective (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.