Research map / Large language models / Instruction and fine-tuning
TuCo: Measuring the Contribution of Fine-Tuning to Individual Responses of LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TuCo, an inference-time metric that decomposes a fine-tuned LLM's residual stream into pre-training and fine-tuning components and reports the fine-tuning share of the last token's representation, showing jailbreaks reduce it.
Problem
Existing analyses of fine-tuning effects are aggregate (benchmark-level) or model-specific; there is no quantitative per-prompt measure of how much fine-tuning contributes to an individual LLM response.
Approach
Exactly decompose each layer's output of a fine-tuned transformer into a pre-training component (the pre-trained layer's output) and a fine-tuning component (fine-tuned minus pre-trained layer output); define TuCo as the norm ratio of accumulated FTC to accumulated PTC+FTC on the last token across all layers, motivated by a discrete Gronwall bound and by FTC-scaling experiments.
Key results
- TuCo is lower on jailbreak-attack prompts: GCG, conjugate prompting and many-shot attacks significantly reduce it across 7 open models, with AUC above 0.78 for four of five GCG models.
- Scaling the fine-tuning component by alpha in 0.75 to 1.25 changes behavior, giving statistically significant MMLU gains for 71% of tasks (1.03% to 2.69% average improvements).
- TuCo separates OpenWebText from HH-RLHF prompts with AUC above 0.80 for all but two models.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase the FTC scaling factor and select it to maximize agreement | FTCα-Scaling · MWE behavioral evaluations | agreement: 1.55% to 5.18% | Helped |
| Select the FTC scaling factor to minimize agreement | FTCα-Scaling · MWE behavioral evaluations | agreement: -2.80% to -25.24% | Hurt |
| Scale the fine-tuning component and optimize the factor per task | FTCα-Scaling · MMLU | accuracy: 1.03% to 2.69% | Helped |
| Apply a GCG attack string to harmful prompts | TuCo on safety-tuned models · Harmful prompts with GCG attacks | TuCo | Hurt |
| Translate harmful prompts into low-resource languages | TuCo on safety-tuned models · Conjugate Prompting | TuCo | Hurt |
| Add harmful behavior examples to the prompt | TuCo on safety-tuned models · Many-Shot Jailbreaking | TuCo | Hurt |
| Use prompts where jailbreak attacks succeed rather than fail | TuCo · Successful jailbreak prompts | AUC for distinguishing successful prompts: 0.87 AUC for Llama 13B | Helped |
| Use low-resource-language prompts rather than English prompts | TuCo on Llama 2 13B · Conjugate Prompting | TuCo | Hurt |
Surprising findings
- Llama 3 8B shows near-zero jailbreak success (0.24%) and an AUC of only 0.51 for predicting successful jailbreaks.
- Output-level similarity (OutputCo) can miss internal fine-tuning activity that TuCo captures, e.g., when output tokens remain unchanged.
Limitations
- TuCo is an analysis tool, not a jailbreak detector, and requires access to both pre-trained and fine-tuned models.
- Jailbreak success is labeled with refusal-keyword matching over 8 samples, which is a coarse success criterion.
- Several evaluated models have unstated or ambiguous parameter counts and the Llama 3 jailbreak results are inconsistent.
Experimental setup
Datasets: OpenWebText HH-RLHF AdvBench MMLU Model-Written Evaluations
Benchmarks: MMLU Model-Written Evaluations AdvBench HH-RLHF OpenWebText
Metrics: AUC accuracy cross-validated accuracy change
Baselines: OutputCo
Builds on: transformer circuits many-shot jailbreaking greedy coordinate gradient conjugate prompting
Closest papers by meaning
- Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation (NeurIPS 2025 · similarity 0.89)
- Understanding Catastrophic Forgetting in Language Models via Implicit Inference (ICLR 2024 · similarity 0.89)
- Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis (ICML 2025 · similarity 0.88)
- When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method (ICLR 2024 · similarity 0.88)
- Learning Dynamics of LLM Finetuning (ICLR 2025 · similarity 0.88)
- Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.