Research map / Large language models / Language, safety and interpretability

Palette: A Modular, Controllable, and Efficient Framework for On-demand Authorized Safety Alignment Relaxation in LLMs

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: Safety alignment of LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Palette selectively relaxes refusal for authorized domains by searching a multi-objective refusal direction, internalizing it into one block via LoRA with hard-negative mining, and composing per-domain adapters by parameter merging.

Problem

One-size-fits-all safety alignment refuses requests that are unsafe for general users but legitimate for authorized professionals, and existing fixes need costly realignment or imprecise inference-time activation steering.

Approach

Select a refusal direction via Pareto-style multi-objective search (control score on allowed/disallowed sets vs. KL utility on safe prompts), internalize it into a single transformer block with LoRA using label-conditioned activation reconstruction, and merge single-domain LoRAs by addition for multi-domain authorization.

Key results

• On WMDP with LLaMA3-8B-Instruct, allowing Chem drops Chem refusal from 0.964 to 0.072 while Cyber refusal stays at 0.987 and average utility stays near base (0.692 vs 0.698).
• CAST reduces Bio refusal to 0.461 when Chem is allowed, versus 0.752 for Palette, showing more safety leakage for CAST.
• On the GenHarm iterative study with LLaMA3.1-8B-Instruct, average response accuracy rises from 0.797 (base) to 0.919 after three refinement iterations.

Surprising findings

• Omitting semantically overlapping negative domains (e.g., Disinformation when allowing Hate) during adaptation causes safety leakage, while omitting distant domains has no effect.

Limitations

• Assumes user authorization is already verified; authorization itself is not addressed.
• Depends on representative allowed and disallowed samples; incomplete coverage can cause leakage to related unseen cases.
• Merging may need conflict resolution for closely related domains or conflicting preferences.

Experimental setup

Datasets: GenHarm WMDP CoSApien MM-SafetyBench AdvBench MaliciousInstruct TDC23-RedTeaming Alpaca HarmBench StrongReject Beavertails JailbreakBench

Benchmarks: GenHarm WMDP CoSApien MM-SafetyBench MMLU GSM8K MMMU MMBench

Metrics: refusal rate response accuracy utility accuracy

Baselines: SFT AutoDAN CAST

Builds on: difference-in-means refusal direction CAST LoRA AutoDAN

Closest papers by meaning

• Expected Harm: Rethinking Safety Evaluation of (Mis)Aligned LLMs (NeurIPS 2026 · similarity 0.88)
• Improving LLM Safety Alignment with Dual-Objective Optimization (ICML 2025 · similarity 0.88)
• SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal (ICLR 2025 · similarity 0.88)
• The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directions (ICML 2025 · similarity 0.88)
• Safety Reasoning with Guidelines (ICML 2025 · similarity 0.88)
• WildGuard: Open One-stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs (NeurIPS 2024 · similarity 0.88)