Research map / Large language models / Instruction and fine-tuning
Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that fine-tuned LLaMA-7B variants (Vicuna, Goat, FLoat) reuse the same entity-tracking circuit and mechanism (tracking the position of the queried entity) as the base model, with the gains coming from better value fetching and positional handling, using path patching, DCM, and a new cross-model activation patching (CMAP).
Problem
How fine-tuning on generic tasks (arithmetic, code, instructions) changes the internal mechanisms of LLMs, using in-context entity tracking as a case study where arithmetic-tuned models perform much better than the base model.
Approach
Identify the entity-tracking circuit in base Llama-7B via path patching, check that the same heads suffice in fine-tuned models, decompose its functionality with Desiderata-based Component Masking, and introduce cross-model activation patching (CMAP) to localize which sub-mechanism fine-tuning improves.
Key results
- The Llama-7B entity-tracking circuit of 72 heads achieves faithfulness 1.0 on Llama-7B and 0.97 on Vicuna-7B, 0.89 on Goat-7B, and 0.88 on FLoat-7B.
- Goat-7B reaches 0.82 full-model accuracy vs 0.66 for Llama-7B; the Value Fetcher heads in Goat-7B improve retrieval by about 20% over Llama-7B.
- Patching Value Fetcher heads from fine-tuned models into Llama-7B recovers the full fine-tuned performance, indicating fine-tuning improves positional-to-object resolution rather than the mechanism itself.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune LLaMA-7B on arithmetic tasks with LoRA | LLaMA-7B · Entity tracking | accuracy: 0.66 -> 0.82 | Helped |
| fine-tune LLaMA-7B on arithmetic tasks without LoRA | LLaMA-7B · Entity tracking | accuracy: 0.66 -> 0.82 | Helped |
| fine-tune LLaMA-7B on user conversations | LLaMA-7B · Entity tracking | accuracy: 0.66 -> 0.67 | No clear effect |
| use the LLaMA-7B entity-tracking circuit in the fine-tuned model | LLaMA-7B circuit on Goat-7B · Entity tracking | accuracy: 0.82 -> 0.73 | Hurt |
| use the LLaMA-7B entity-tracking circuit in the fine-tuned model | LLaMA-7B circuit on FLoat-7B · Entity tracking | accuracy: 0.82 -> 0.72 | Hurt |
| use the Goat-7B circuit instead of the full model | LLaMA-7B · Entity tracking | accuracy: 0.66 -> 0.77 | Helped |
| use the Goat-7B circuit instead of the full model | Vicuna-7B · Entity tracking | accuracy: 0.67 -> 0.76 | Helped |
| use the Goat-7B circuit on its source model | Goat-7B · Entity tracking | accuracy: 0.82 -> 0.81 | No clear effect |
| use the FLoat-7B circuit instead of the full model | LLaMA-7B · Entity tracking | accuracy: 0.66 -> 0.69 | Helped |
| use the FLoat-7B circuit instead of the full model | Vicuna-7B · Entity tracking | accuracy: 0.67 -> 0.7 | Helped |
| use the FLoat-7B circuit on its source model | FLoat-7B · Entity tracking | accuracy: 0.82 -> 0.82 | No clear effect |
Surprising findings
- Goat-7B and FLoat-7B circuits in the fine-tuned models contain about 175 heads, a superset of the Llama-7B circuit, yet the shared heads implement the same functionality.
- The Goat-7B circuit gives higher accuracy than the full model when run on Llama-7B and Vicuna-7B (faithfulness above 1.0), attributed to negative heads.
Limitations
- Only one task (entity tracking) and three fine-tuned LLaMA-7B variants are studied, so mechanism invariance may not generalize.
- The Llama-7B circuit is not perfectly complete; additional or backup heads contribute, and fine-tuned circuits are much larger (175 heads).
Experimental setup
Datasets: synthetic box tracking dataset (adapted from prior entity tracking dataset) synthetic arithmetic expressions
Benchmarks: synthetic box tracking entity tracking task
Metrics: accuracy faithfulness completeness minimality
Baselines: Llama-7B Vicuna-7B Goat-7B FLoat-7B
Builds on: path patching Desiderata-based Component Masking LoRA tracr
Closest papers by meaning
- Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis (ICML 2025 · similarity 0.88)
- An Emulator for Fine-tuning Large Language Models using Small Language Models (ICLR 2024 · similarity 0.87)
- Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks (ICLR 2024 · similarity 0.87)
- From Style to Facts: Mapping the Boundaries of Knowledge Injection with Finetuning (NeurIPS 2025 · similarity 0.86)
- A Closer Look at the Limitations of Instruction Tuning (ICML 2024 · similarity 0.86)
- Understanding Catastrophic Forgetting in Language Models via Implicit Inference (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.