Research map / Large language models / Language, safety and interpretability
MoGU: A Framework for Enhancing Safety of LLMs While Preserving Their Usability
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MoGU, which turns an LLM into a usable and a safe LoRA variant and mixes them with a learned dynamic router so that malicious inputs favor the safe variant and benign inputs the usable one.
Problem
Existing LLM safety defenses make models reject benign instructions too often, lowering response usability; the goal is to raise safety against malicious and jailbreak prompts while keeping helpfulness on benign ones.
Approach
Build two LoRA variants of the base model (a Glad Responder trained to answer everything and an Unwilling Responder trained to refuse everything), then train a per-layer sigmoid router on the O_proj outputs that weights them by input type, using contrastive-style negative samples and an L1 constraint on weights; only the first few decoded tokens use MoGU.
Key results
- On AdvBench/Malicious red-team evaluation for Llama2, Vicuna and Falcon, MoGU averages 3.15% ASR, second only to SafeDecoding's 1.42%.
- On Llama2-7B, ICD raises the benign rejection rate from 14.00% to 92.25% and drops GPT-Eval average from 3.87 to 2.17, while MoGU keeps 16.50% rejection and 3.87 GPT-Eval.
- MoGU adds about 3.91% parameters to Llama2-7B (273,678,336 added parameters) and decodes only the first 5 tokens with the MoGU mixture.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply MoGU instead of No Defense | Dolphin-7B · Red-team and jailbreak AVG. | ASR: 93.98% -> 20.81% | Helped |
| apply MoGU instead of No Defense | Baichuan2-7B · Red-team and jailbreak AVG. | ASR: 23.91% -> 4.10% | Helped |
| apply MoGU instead of No Defense | Llama2-7B · Just-Eval benign | GPT-Eval AVG.: 3.87 -> 3.87 | No clear effect |
| apply MoGU instead of No Defense | Llama2-7B · Just-Eval benign | rejection expressions: 14.00% -> 16.50% | Hurt |
| add contrastive learning loss (ablation) | MoGU · Red-team and jailbreak AVG., Llama2-7B | ASR: 1.50% -> 0.29% | Helped |
| add L1 Norm constraint (ablation) | MoGU · Red-team and jailbreak AVG., Llama2-7B | ASR: 4.49% -> 0.29% | Helped |
| add contrastive learning loss (ablation) | MoGU · Red-team and jailbreak AVG., Vicuna-7B | ASR: 9.64% -> 2.36% | Helped |
| add L1 Norm constraint (ablation) | MoGU · Red-team and jailbreak AVG., Vicuna-7B | ASR: 33.36% -> 2.36% | Helped |
| add contrastive learning loss (ablation) | MoGU · Red-team and jailbreak AVG., Falcon-7B | ASR: 10.13% -> 10.49% | Hurt |
| add L1 Norm constraint (ablation) | MoGU · Red-team and jailbreak AVG., Falcon-7B | ASR: 23.10% -> 10.49% | Helped |
| set router dimension to 512 rather than 128, 256, or 1024 (ablation) | MoGU · Red-team and jailbreak evaluations on Llama2-7B, Vicuna-7B, and Falcon-7B | ASR | Helped |
Surprising findings
- Ablation shows removing the L1 norm constraint hurts defense more than removing the contrastive loss (e.g., Vicuna average ASR rises from 2.36% to 33.36%).
Limitations
- Only the O_proj linear layer was explored as the LoRA target; other layers are untested.
- Introduced parameters are not shown to be all necessary; pruning was not explored.
- MoGU is weaker than SafeDecoding on red-team ASR for Falcon (17.50% on Malicious) and the GPT-4 judge is the main safety metric.
Experimental setup
Datasets: AdvBench Alpaca Just-Eval
Benchmarks: AdvBench Malicious (Just-Eval) AutoDAN GCG PAIR SAP30 Comp_obj Just-Eval
Metrics: ASR GPT-Judge harmfulness score GPT-Eval (helpfulness, clarity, factuality, depth, engagemen rule-based rejection rate
Baselines: SFT Detect_inp Self-Examine Retokenization Self-Reminder ICD SafeDecoding
Builds on: LoRA Mixture of Experts SFT SafeDecoding In-Context Defense
Closest papers by meaning
- WildGuard: Open One-stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs (NeurIPS 2024 · similarity 0.89)
- On Evaluating the Durability of Safeguards for Open-Weight LLMs (ICLR 2025 · similarity 0.89)
- Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions (ICLR 2024 · similarity 0.88)
- Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions (ICML 2025 · similarity 0.88)
- SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal (ICLR 2025 · similarity 0.88)
- STAIR: Improving Safety Alignment with Introspective Reasoning (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.