Research map / Large language models / Instruction and fine-tuning
Watch the Weights: Unsupervised monitoring and control of fine-tuned LLMs
Ziqian Zhong, Aditi Raghunathan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use weight-difference singular directions for monitoring instead of BEAT | backdoor detection on PPO trojans · PPO trojan3 TPR | TPR: 31.95 -> 100.00 | Improved |
| use weight-difference directions instead of activation difference norm baseline | backdoor detection on PPO trojans · PPO trojan3 TPR | TPR: 11.50 -> 100.00 | Improved |
| weight-based monitoring on LoRA mtba backdoor | WeightWatch · LoRA mtba | % prompts flagged: 27.27 (vs 100.00 on other triggers) | Hurt |
| use different trigger words for mtba LoRA replication | WeightWatch · LoRA mtba | % prompts flagged: 27.27 -> 52.53 | Improved |
| regenerate cleaner FFT backdoor dataset | WeightWatch FFT backdoor models · FFT ctba | % prompts flagged: 38 -> 100.00 | Improved |
| steer along weight directions with default calibration range | Zephyr-RMU · WMDP-Bio | accuracy: 31.2 -> 39.7 | Improved |
| widen steering range (drop top/bottom 0.1% calibration), FPR 0.94% -> 8.54% (ablation) | Zephyr-RMU steering · WMDP-Bio | accuracy: 39.7 -> 56.2 | Improved |
| widen steering range (FPR 8.54%) (ablation) | Zephyr-RMU steering · MMLU | accuracy: 57.1 -> 57.0 | No clear effect |
| steer along directions plus prompt prefix | Circuit Breaker · HarmBench | attack success rate: 9.6 -> 82.08 | Improved |
| steer along weight directions | WHP unlearned model · knowledge recovery | qualitative | No clear effect |
| more fine-tuning data on math in OLMo SFT/DPO | direction counts · math directions | % directions: 15 (1.5%) directions despite 4.84%-7.74% math data | No clear effect |
Closest papers by meaning
- Keeping an Eye on LLM Unlearning: The Hidden Risk and Remedy (NeurIPS 2025 · similarity 0.90)
- WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Models (NeurIPS 2024 · similarity 0.89)
- OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics (NeurIPS 2025 · similarity 0.89)
- Simulate and Eliminate: Revoke Backdoors for Generative Large Language Models (AAAI 2025 · similarity 0.89)
- Representation Noising: A Defence Mechanism Against Harmful Finetuning (NeurIPS 2024 · similarity 0.89)
- BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
