Research map / Large language models / Alignment and preferences
Configurable Reward Model for Balanced Safety Alignment
Zhengping Jiang, Mehran Khodabandeh, Akash Bharadwaj, Manik Bhandari, Mayur Srungarapu, Anqi Liu, Benjamin Van Durme, Li Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add configurable safety configuration augmentation (ablation) | CSRM · CoSApien | F1: 0.815 -> 0.946 | Improved |
| add configurable safety configuration augmentation (ablation) | CSRM · DynaBench | F1: 0.692 -> 0.758 | Improved |
| add configurable safety configuration augmentation (ablation) | CSRM · BeaverTails | smECE: 0.049 -> 0.017 | Improved |
| add configurable safety configuration augmentation (ablation) | CSRM · WildGuardMix | F1: 0.953 -> 0.953 | No clear effect |
| add severity augmentation (ablation) | CSRM (-CCA) · BeaverTails-Aug pairwise | accuracy: 0.583 -> 0.762 | Improved |
| add severity augmentation (ablation) | CSRM (-CCA) · CoSApien | smECE: 0.141 -> 0.123 | Improved |
| add severity augmentation (ablation) | CSRM (-CCA) · Safety-RLHF pairwise | accuracy: 0.661 -> 0.763 | Improved |
| use CSRM as reward instead of Llama-3.1-8B-Inst | Reinforce++ alignment · Arab Publisher | CoSA score: 3.475 -> 3.603 | Improved |
| use CSRM as reward instead of LlamaGuard-3-8B | Reward Distillation · Arab Publisher | CoSA score: 2.825 -> 3.145 | Improved |
| increase reward-loss margin from 0.0 to 1.0 (ablation) | CSRM (Qwen3-2B) · CoSApien | F1: 0.890 -> 0.757 | Hurt |
| increase reward-loss margin from 0.0 to 1.0 (ablation) | CSRM (Qwen3-2B) · BeaverTails | smECE: 0.055 -> 0.070 | Hurt |
| replace strict policy with Allow policy for target category (ablation) | CSRM · BeaverTails single-category | unsafe rate: 0.843 -> lowered by 0.708 | Improved |
Closest papers by meaning
- Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements (ICLR 2025 · similarity 0.90)
- STAIR: Improving Safety Alignment with Introspective Reasoning (ICML 2025 · similarity 0.89)
- AlphaAlign: Incentivizing Safety Alignment with Extremely Simplified Reinforcement Learning (ICLR 2026 · similarity 0.89)
- Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neurons (NeurIPS 2025 · similarity 0.88)
- Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (ICLR 2024 · similarity 0.88)
- BSO: Safety Alignment Is Density Ratio Matching (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
