Research map / Large language models / Language, safety and interpretability
Do Activation Verbalization Methods Convey Privileged Information?
Millicent Li, Alberto Mario Ceballos Arroyo, Giordano Rogers, Naomi Saphra, Byron C Wallace
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| give verbalizer activations from target model instead of text only | LIT (Llama3) · Feature extraction average | accuracy: 0.64 vs 0.64 zero-shot | No clear effect |
| give verbalizer activations instead of text only | Patchscopes (Llama3) · Feature extraction average | accuracy: 0.38 vs 0.64 zero-shot | Hurt |
| give verbalizer activations instead of text only | Patchscopes (Ministral) · Feature extraction average | accuracy: 0.27 vs 0.68 zero-shot | Hurt |
| use different model families for target and verbalizer | LIT · Feature extraction country-currency | accuracy: 0.79 -> 0.00 | Hurt |
| replace activations with reconstructed input text via inversion (ablation) | Llama3 verbalizer · Feature extraction, all activations | accuracy: 0.79 vs 0.79 (country-currency) | No clear effect |
| invert single token layer-15 activation instead of verbalizing it (ablation) | Patchscopes (layer 15) · Feature extraction single token | accuracy: 0.16 -> 0.32 (athlete-sport) | Improved |
| invert all activations instead of using LIT (ablation) | LIT · Feature extraction athlete-sport | accuracy: 0.84 -> 0.39 | Hurt |
| shuffle persona attributes to remove sociodemographic correlation (ablation) | LIT · PersonaQA country | accuracy: 0.72 -> 0.03 | Hurt |
| use fantasy personas unknown to verbalizer (ablation) | LIT and Patchscopes · PersonaQA-Fantasy | accuracy: 0.00 on all attributes | Hurt |
| finetune verbalizer M2 on the same persona knowledge as M1 (ablation) | LIT · PersonaQA-Fantasy country | accuracy: 0.00 -> 0.20 | Improved |
| verbalizer has knowledge the target lacks (ablation) | LIT · PersonaQA-Fantasy fav food | accuracy: 0.45 non-zero despite M1 lacking knowledge | Hurt |
| compare linear probe to LIT on M1 activations (ablation) | Logistic probe · PersonaQA-Fantasy | accuracy: 0.38 vs 0.25 (fav food) | No clear effect |
Closest papers by meaning
- Training Language Models to Explain Their Own Computations (NeurIPS 2026 · similarity 0.87)
- Looking Inward: Language Models Can Learn About Themselves by Introspection (ICLR 2025 · similarity 0.87)
- Exploring Activation Patterns of Parameters in Language Models (AAAI 2025 · similarity 0.86)
- Do LLMs ``know'' internally when they follow instructions? (ICLR 2025 · similarity 0.86)
- Sparsity-Guided Holistic Explanation for LLMs with Interpretable Inference-Time Intervention (AAAI 2024 · similarity 0.86)
- LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
