Research map / Large language models / Language, safety and interpretability
ProSafePrune: Projected Safety Pruning for Mitigating Over-Refusal in LLMs
Zijun Chen, Wenbo Hu, Ya Li, Lei Miao, Guoping Hu, Richang Hong
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply ProSafePrune low-rank pruning | LLaMA-2-7B · OR-Bench-Hard-1K | compliance rate: 11.0 -> 73.0 | Improved |
| apply ProSafePrune low-rank pruning | LLaMA-2-13B · average trade-off score | Avg. T.S.: 65.8 -> 82.5 | Improved |
| apply ProSafePrune low-rank pruning | LLaMA-2-70B · OR-Bench-Hard-1K | compliance rate: 6.5 -> 68.5 | Improved |
| apply ProSafePrune low-rank pruning | Qwen-2.5-7B · OR-Bench-Hard-1K | compliance rate: 83.5 -> 91.0 | Improved |
| apply ProSafePrune low-rank pruning | LLaMA-2-7B · MMLU | accuracy: 37.1 -> 39.6 | Improved |
| apply ProSafePrune low-rank pruning | LLaMA-2-7B · GSM8K | accuracy: 23.0 -> 25.5 | Improved |
| apply ProSafePrune low-rank pruning | Qwen-2.5-7B · MMLU | accuracy: 82.8 -> 81.0 | No clear effect |
| prune the whole layer instead of only V projection (ablation) | ProSafePrune · OR-Bench-Hard-1K (LLaMA-2-7B) | compliance rate: 30.5 -> 73.0 | Improved |
| add pseudo-harmful projection Pi(p) to the overlap operator (ablation) | ProSafePrune · LLaMA-2-7B safety (AdvBench, JBB) | safety score: 79.5 -> 98.5; 71.0 -> 94.0 | Improved |
| add pseudo-harmful projection Pi(p) to the overlap operator (ablation) | ProSafePrune · LLaMA-2-7B over-refusal (OR-Bench, PHTest) | compliance rate: 90.5 -> 73.0; 96.5 -> 94.5 | Hurt |
| prune middle layers [13,16] instead of bottom layers (ablation) | ProSafePrune · OR-Bench-Hard-1K (LLaMA-2-7B) | compliance rate: 14.0 -> 73.0 | Improved |
| prune middle layers [13,16] instead of top layers (ablation) | ProSafePrune · OR-Bench-Hard-1K (LLaMA-2-7B) | compliance rate: 13.0 -> 73.0 | Improved |
Closest papers by meaning
- LLMs Encode Harmfulness and Refusal Separately (NeurIPS 2025 · similarity 0.92)
- Just Enough Shifts: Mitigating Over-Refusal in Aligned Language Models with Targeted Representation Fine-Tuning (ICML 2025 · similarity 0.91)
- OR-Bench: An Over-Refusal Benchmark for Large Language Models (ICML 2025 · similarity 0.91)
- The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence (ICML 2025 · similarity 0.91)
- Fail-Closed Alignment for Large Language Models (NeurIPS 2026 · similarity 0.91)
- Latent-space Attacks for Refusal Evasion in Language Models (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
