Research map / Large language models / Language, safety and interpretability
The Art of Saying No: Contextual Noncompliance in Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CoCoNot, a taxonomy-driven benchmark (1,000 eval prompts) and synthetic training set for contextual noncompliance beyond safety, benchmarks many LMs, and finds LoRA continued finetuning of Tulu-2 improves noncompliance while preserving general capabilities.
Problem
Chat LMs are trained to be helpful but should sometimes decline requests beyond unsafe content, such as incomplete, unsupported, indeterminate, and humanizing requests; existing work focuses mainly on safety refusals and models show high compliance on these other categories.
Approach
A five-category contextual noncompliance taxonomy with subcategory-specific acceptable behaviors, used to build the CoCoNot evaluation set and a synthetic training set with contrast (should-comply) queries; training compares SFT from scratch, continued SFT, LoRA, and DPO on contrast preference data.
Key results
- GPT-4 complies with 29.8% of incomplete requests and Llama-2-70b with 40.8% of unsupported requests in the no-system-prompt setting, showing high compliance in understudied categories.
- LoRA continued finetuning of Tulu-2 7B on CoCoNot strongly reduces compliance on CoCoNot categories while keeping MMLU (50.0) and AlpacaEval (74.2) close to the base model.
- Merging the LoRA adapter trained on Tulu-2-no-refusal into Tulu-2 yields better results than both LoRA-tuned models, reportedly outperforming GPT-4 compliance rates on CoCoNot.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add CoCoNot training data through SFT from the pretrained Llama-2 model | Llama-2 7B · CoCoNot Incomplete | compliance rate: 25.8 -> 5.3 | Helped |
| Continue training all parameters on CoCoNot rather than using LoRA (ablation) | Tulu-2 7B · CoCoNot Contrast | compliance rate: 90.8 -> 31.4 | Hurt |
| Add matched-sized T2M samples during continued full-parameter SFT (ablation) | Tulu-2 7B · CoCoNot Contrast | compliance rate: 31.4 -> 54.9 | Helped |
| Use LoRA for continued SFT on CoCoNot instead of full-parameter continued SFT (ablation) | Tulu-2 7B · MMLU-0 | EM: 48.0 -> 50.0 | Helped |
| Use LoRA for continued SFT on CoCoNot (ablation) | Tulu-2 7B · CoCoNot Contrast | compliance rate: 31.4 -> 90.8 | Helped |
| Merge the adapter trained on Tulu-2-no-refusal with Tulu-2 (ablation) | Tulu-2-no-refusal 7B LoRA · CoCoNot Unsupported | compliance rate: 39.4 -> 12.8 | Helped |
| DPO on CoCoNot preference data to prefer compliant answers to benign queries | Tulu-2 7B LoRA · CoCoNot Contrast | compliance rate: 90.8 -> 93.1 | Helped |
| Add a system prompt instructing noncompliance | GPT-4 · CoCoNot Incomplete | compliance rate: 29.8 / 19.6 | Helped |
| Add a system prompt instructing noncompliance | GPT-4o · CoCoNot Incomplete | compliance rate: 8.9 / 30.2 | Hurt |
Surprising findings
- A system prompt instructing noncompliance does not reliably help and sometimes increases compliance (e.g., GPT-4o incomplete requests rise from 8.9% to 30.2%).
- Training from scratch with CoCoNot data mixed into Tulu2Mix leads to overrefusal, with contrast-set compliance dropping to 74.9%.
Limitations
- Dataset is mostly synthetic generated by GPT models and may be noisy; only evaluation split manually verified.
- Training experiments limited to Llama-2 7B base due to compute budget.
- Taxonomy is US-centric and not every category demands noncompliance for every model.
Experimental setup
Datasets: CoCoNot Tulu2Mix SituatedQA WildChat XSTest HarmBench ToxiGen MMLU AlpacaEval BBH GSM8k TydiQA
Benchmarks: CoCoNot XSTest HarmBench ToxiGen AlpacaEval MMLU BBH GSM8k TydiQA TruthfulQA Codex-Eval
Metrics: compliance rate attack success rate toxic output percentage accuracy win rate
Baselines: GPT-4 GPT-4o Claude-3 Sonnet Llama-3-70b Llama-2-70b Mixtral Mistral Tulu-2-70b-dpo Vicuna
Builds on: Self-Instruct XSTest Tulu 2 DPO LoRA
Closest papers by meaning
- SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal (ICLR 2025 · similarity 0.88)
- OR-Bench: An Over-Refusal Benchmark for Large Language Models (ICML 2025 · similarity 0.88)
- Measuring Non-Adversarial Reproduction of Training Data in Large Language Models (ICLR 2025 · similarity 0.87)
- On the Worst Prompt Performance of Large Language Models (NeurIPS 2024 · similarity 0.87)
- Refusal in Language Models Is Mediated by a Single Direction (NeurIPS 2024 · similarity 0.87)
- CASE-Bench: Context-Aware SafEty Benchmark for Large Language Models (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.