Research map / Large language models / Agents, code and math
Improving Reasoning Performance in Large Language Models via Representation Engineering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives representation-engineering control vectors from residual-stream activations on reasoning tasks (IOI, bAbI, GSM8K) and applies them at inference to modestly improve accuracy of Pythia and Mistral models without training.
Problem
Whether reasoning ability in LLMs is encoded in the residual stream such that it can be modulated to improve performance on reasoning tasks at inference time.
Approach
Extract residual-stream activations on successful vs. unsuccessful reasoning examples, derive a layer-wise control vector (mean difference or PCA direction), and add it at the middle layer during inference with scaling alpha, improving task accuracy without any weight training.
Key results
- Control vectors improve Mistral-7B-Instruct accuracy on GSM8K with negative alpha, with bAbI-derived vectors giving an almost identical gain on GSM8K.
- Pythia-2.8B shows slight accuracy improvements on IOI across most conditions when PCA control vectors are applied.
- KL divergence increases roughly quadratically with alpha, while accuracy gains are modest and the change in probability on correct tokens is minute for Pythia.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply PCA-derived residual-stream control vectors at the middle layer | Pythia-1.4B and Pythia-2.8B · IOI | logit-based accuracy | Helped |
| Apply PCA-derived residual-stream control vectors at the middle layer | Pythia-2.8B · IOI | logit-based accuracy | Helped |
| Apply a contrast-based control vector at the middle layer | Mistral-7B-Instruct · bAbI task 15 | logit-based accuracy | Helped |
| Apply a PCA-contrast control vector derived from GSM8K | Mistral-7B-Instruct · GSM8K | accuracy | Helped |
| Apply a control vector derived from bAbI | Mistral-7B-Instruct · GSM8K | accuracy | Helped |
| Apply a control vector derived from GSM8K | Mistral-7B-Instruct · bAbI task 15 | accuracy | Helped |
| Use positive α rather than negative α (ablation) | Pythia-1.4B control vectors · IOI | accuracy | Mixed |
| Use random-character-string negative prompts rather than reading vectors (ablation) | Reasoning control-vector extraction · Reasoning tasks | performance | Helped |
| Apply the IOI-derived control vector across IOI conditions | Pythia models · IOI conditions B, ABBA-Long, and BABA-Long | accuracy | Helped |
Surprising findings
- A control vector derived from bAbI improves GSM8K accuracy, suggesting cross-task transfer of the intervention.
- Entropy decreases do not always coincide with accuracy gains, and for Mistral on bAbI entropy trends upward despite accuracy improvements.
Limitations
- Only small models (Pythia up to 2.8B) and one 7B model; improvements are modest and results are unstable across alpha.
- IOI is too simple and Pythia models could not solve bAbI or GSM8K, limiting evaluation.
- Choice of contrastive negatives (random character strings) is ad hoc; no principled method for deriving optimal control vectors.
Experimental setup
Datasets: IOI (synthetic) bAbI task 15 GSM8K
Benchmarks: IOI bAbI GSM8K
Metrics: logit-based accuracy KL divergence entropy average probability on correct answer
Baselines: reading vectors random contrastive pairs
Builds on: representation engineering contrastive activation addition
Closest papers by meaning
- Training Language Models to Reason Efficiently (NeurIPS 2025 · similarity 0.91)
- Calibrating Reasoning in Language Models with Internal Consistency (NeurIPS 2024 · similarity 0.90)
- Who Reasons in the Large Language Models? (NeurIPS 2025 · similarity 0.89)
- Longer Context, Deeper Thinking: Uncovering the Role of Long-Context Ability in Reasoning (NeurIPS 2025 · similarity 0.89)
- The Tell-Tale Norm: $\ell_2$ Magnitude as a Signal for Reasoning Dynamics in Large Language Models (ICML 2026 · similarity 0.89)
- Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.