Research map / Large language models / Retrieval and knowledge
Learning to Interpret Weight Differences in Language Models
Avichal Goel, Yoon Kim, Nir N Shavit, Tony T. Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DIT adapter instead of Base Question | Qwen3-4B · Hidden topic reporting (SEP trigger) | similarity score: 1.21 -> 4.76 | Improved |
| use DIT adapter instead of 20 Questions | Qwen3-4B · Hidden topic reporting (SEP trigger) | similarity score: 1.38 -> 4.76 | Improved |
| evaluate DIT on OOD zero-width space trigger at start (ablation) | DIT adapter · Hidden topic reporting | similarity score: 4.76 -> 4.66 | No clear effect |
| evaluate DIT on OOD zero-width space trigger at random position (ablation) | DIT adapter · Hidden topic reporting | similarity score: 4.76 -> 4.58 | No clear effect |
| add 20 questions probing without DIT on OOD random-position trigger (ablation) | 20 Questions · Hidden topic reporting | similarity score: 1.38 -> 2.31 | Improved |
| train on topic weight diffs, test on news summary weight diffs (Qwen3-4B) (ablation) | Topic Adapter · Cross-behavior generalization | similarity score: 4.22 -> 1.25 | Hurt |
| train on news weight diffs, test on hidden topic weight diffs (Qwen3-4B) (ablation) | News Adapter · Cross-behavior generalization | similarity score: 4.76 -> 1.22 | Hurt |
| train on topic weight diffs, test on news summary weight diffs (Gemma3-4B) (ablation) | Topic Adapter · Cross-behavior generalization | similarity score: 4.56 -> 1.08 | Hurt |
| train on news weight diffs, test on hidden topic weight diffs (Gemma3-4B) (ablation) | News Adapter · Cross-behavior generalization | similarity score: 4.82 -> 1.16 | Hurt |
| train DIT to predict the SEP trigger instead of hidden topic | DIT adapter · Trigger recovery | correct triggers: 0 out of 100 | Hurt |
| train DIT adapter on rank-1 LoRA diffs, test on higher rank and full-parameter diffs (ablation) | DIT adapter · Hidden topic reporting | similarity score | Mixed |
| increase DIT training data amount (ablation) | DIT adapter · Hidden topic reporting | similarity score | Improved |
Closest papers by meaning
- An Emulator for Fine-tuning Large Language Models using Small Language Models (ICLR 2024 · similarity 0.89)
- When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method (ICLR 2024 · similarity 0.87)
- Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks (ICLR 2024 · similarity 0.87)
- DELIFT: Data Efficient Language model Instruction Fine-Tuning (ICLR 2025 · similarity 0.87)
- Overcoming Sparsity Artifacts in Crosscoders to Interpret Chat-Tuning (NeurIPS 2025 · similarity 0.86)
- Data Efficient Adaptation in Large Language Models via Continuous Low-Rank Fine-Tuning (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
