Research map / Large language models / Instruction and fine-tuning
Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuning
Helena Casademunt, Caden Juang, Adam Karvonen, Samuel Marks, Senthooran Rajamanoharan, Neel Nanda
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add CAFT with PCA ablation during fine-tuning | insecure-code fine-tuning (Qwen) · Emergent misalignment (Qwen) | misaligned response rate (%): 7.0% to 0.51% | Improved |
| add CAFT with PCA ablation during fine-tuning | insecure-code fine-tuning (Mistral) · Emergent misalignment (Mistral) | misaligned response rate (%): 6.6% to 1.2% | Improved |
| add CAFT with SAE latent ablation during fine-tuning | insecure-code fine-tuning (Qwen) · Emergent misalignment (Qwen) | misaligned response rate (%): 7.00 -> 2.38 | Improved |
| add CAFT with SAE latent ablation during fine-tuning | insecure-code fine-tuning (Mistral) · Emergent misalignment (Mistral) | misaligned response rate (%): 6.57 -> 4.63 | Improved |
| ablate random orthogonal vectors instead of interpreted directions (ablation) | CAFT (Qwen) · Emergent misalignment (Qwen) | misaligned response rate (%): 7.00 -> 7.88 | No clear effect |
| select interpreted PCs rather than random PCs (ablation) | CAFT with PCA (Qwen) · Emergent misalignment (Qwen) | misaligned response rate (%): 0.87 -> 0.51 misaligned; 58.1 -> 84.8 vulnerable | Improved |
| select interpreted PCs rather than random PCs (ablation) | CAFT with PCA (Mistral) · Emergent misalignment (Mistral) | misaligned response rate (%): 4.47 -> 1.16 | Improved |
| use interpreted SAE latents rather than random SAE latents (ablation) | CAFT with SAE (Mistral) · Emergent misalignment (Mistral) | misaligned response rate (%): 4.56 -> 4.63 | No clear effect |
| ablate top PCs instead of interpreted PCs (ablation) | CAFT with PCA (Mistral) · Emergent misalignment (Mistral) | misaligned response rate (%): 0.34 vs 1.16 | Hurt |
| add CAFT with SAE latents on spurious-correlation tasks | Gemma-2-2B fine-tuning · Gender bias (OOD accuracy) | accuracy: 0.08 -> 0.76 | Improved |
| add CAFT with PCA on spurious-correlation tasks | Gemma-2-2B fine-tuning · Gender bias (OOD accuracy) | accuracy: 0.08 -> 0.64 | Improved |
| add CAFT with SAE latents on double multiple choice | Gemma-2-2B fine-tuning · Double multiple choice (verbs | sports) | accuracy: 0.01 -> 0.99 | Improved |
Closest papers by meaning
- Two-stage LLM Fine-tuning with Less Specialization and More Generalization (ICLR 2024 · similarity 0.91)
- Synergy over Discrepancy: A Partition-Based Approach to Multi-Domain LLM Fine-Tuning (NeurIPS 2025 · similarity 0.88)
- DELIFT: Data Efficient Language model Instruction Fine-Tuning (ICLR 2025 · similarity 0.88)
- FisherSFT: Data-Efficient Supervised Fine-Tuning of Language Models Using Information Gain (ICML 2025 · similarity 0.88)
- Get more for less: Principled Data Selection for Warming Up Fine-Tuning in LLMs (ICLR 2024 · similarity 0.87)
- When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
