Research map / Large language models / Instruction and fine-tuning
Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes how EAP-IG circuits evolve during LoRA/full fine-tuning of Pythia-1.4B on arithmetic and math tasks, finds edges change more than nodes, and uses these edge-change signals to allocate LoRA ranks (CircuitLoRA) with gains over standard LoRA.
Problem
Fine-tuning improves LLMs but its internal mechanisms are poorly understood, especially on tasks where pre-trained models initially perform poorly. Prior circuit studies focus on tasks where the base model already performs well.
Approach
Build arithmetic/math tasks where fine-tuning yields large gains, track EAP-IG circuit node and edge changes across checkpoints, and use layers with the most edge changes to assign higher LoRA ranks (CircuitLoRA); also test whether the union of subtask circuits approximates a compositional task circuit.
Key results
- CircuitLoRA (r_o=16, r_c=64, 1.42% params) reaches 83.10% on Add/Sub(300) vs 80.50% for LoRA (r_o=32, 1.75% params) and 79.10% for RandomLoRA; average gain reported as 2.46% over LoRA.
- Edge change rates exceed node change rates by about 2-4x across five tasks (e.g., LCM edge 80.4% vs node 23.9%).
- On the two-step compositional task, CircuitLoRA with union circuit reaches 65.50% vs 67.20% with compositional circuit and 64.70% for the best LoRA baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Allocate higher LoRA ranks to circuit-identified critical layers (r_o=8, r_c=32) instead of uniform LoRA r=16 | Pythia-1.4B · Add/Sub(300) | accuracy: 78.40 -> 82.70 | Helped |
| Allocate higher LoRA ranks to circuit-identified critical layers (r_o=8, r_c=32) instead of uniform LoRA r=16 | Pythia-1.4B · Mul/Div | accuracy: 95.50 -> 96.00 | Helped |
| Allocate higher LoRA ranks to circuit-identified critical layers (r_o=8, r_c=32) instead of uniform LoRA r=16 | Pythia-1.4B · Sequence | accuracy: 83.40 -> 92.20 | Helped |
| Allocate higher LoRA ranks to circuit-identified critical layers (r_o=8, r_c=32) instead of uniform LoRA r=16 | Pythia-1.4B · LCM | accuracy: 91.20 -> 92.60 | Helped |
| Allocate higher LoRA ranks to circuit-identified critical layers (r_o=8, r_c=32) instead of uniform LoRA r=16 | Pythia-1.4B · Function | accuracy: 97.30 -> 99.40 | Helped |
| Use union subtask circuits rather than the compositional circuit to identify critical layers (ablation) | CircuitLoRA on Pythia-1.4B · Two-Step Operations | accuracy: 67.20 -> 65.50 | Hurt |
| Use union circuit for critical-layer identification rather than randomly selected critical layers | CircuitLoRA on Pythia-1.4B · Two-Step Operations | accuracy: 62.30 -> 65.50 | Helped |
| Increase CircuitLoRA ranks from (r_o=8, r_c=32) to (r_o=16, r_c=64) (ablation) | Pythia-1.4B · Sequence | accuracy: 92.20 -> 94.60 | Helped |
| Increase CircuitLoRA ranks from (r_o=8, r_c=32) to (r_o=16, r_c=64) (ablation) | Pythia-1.4B · Add/Sub(300) | accuracy: 82.70 -> 83.10 | Helped |
| Increase CircuitLoRA ranks from (r_o=8, r_c=32) to (r_o=16, r_c=64) (ablation) | Pythia-1.4B · Mul/Div | accuracy: 96.00 -> 97.00 | Helped |
| Increase CircuitLoRA ranks from (r_o=8, r_c=32) to (r_o=16, r_c=64) (ablation) | Pythia-1.4B · LCM | accuracy: 92.60 -> 93.00 | Helped |
| Increase CircuitLoRA ranks from (r_o=8, r_c=32) to (r_o=16, r_c=64) (ablation) | Pythia-1.4B · Function | accuracy: 99.40 -> 99.50 | Helped |
Surprising findings
- Pre-trained models with very low accuracy still yield circuits with high faithfulness and robustness comparable to fine-tuned models.
- Edges, not nodes, drive most structural change during fine-tuning, contrary to prior claims that fine-tuning only adds components.
Limitations
- Experiments limited to synthetic arithmetic and simple math tasks, not general reasoning.
- Only two-step compositional arithmetic tested.
- Gains are modest (around 2.46% average) and based on five seeds with small models.
Experimental setup
Datasets: Synthetic arithmetic tasks (Add/Sub, Mul/Div, Sequence, LCM, Function) and two-step compositional arithmetic
Benchmarks: Add/Sub (100-500) Mul/Div Sequence LCM Function evaluation two-step compositional arithmetic
Metrics: accuracy faithfulness (KL-based) robustness (edge Jaccard similarity) node/edge change rate top-k overlap
Baselines: LoRA RandomLoRA Full FT AdaLoRA
Builds on: EAP-IG LoRA AdaLoRA Automated Circuit Discovery
Closest papers by meaning
- Linearization Explains Fine-Tuning in Large Language Models (NeurIPS 2025 · similarity 0.89)
- LLM Circuit Analyses Are Consistent Across Training and Scale (NeurIPS 2024 · similarity 0.89)
- LensLLM: Unveiling Fine-Tuning Dynamics for LLM Selection (ICML 2025 · similarity 0.89)
- Transcoders find interpretable LLM feature circuits (NeurIPS 2024 · similarity 0.88)
- Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking (ICLR 2024 · similarity 0.88)
- Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.