Research map / Large language models / Language, safety and interpretability
Single Character Perturbations Break LLM Alignment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that appending a single space (or other single character) to the chat template causes most open-source chat models to produce harmful outputs, and attributes this to space's contexts in pre-training data (tokenization effects toward lists).
Problem
Chat LLMs are aligned to refuse unsafe prompts, but their safety may be fragile to tiny, untargeted changes in the conversation template, such as a trailing space.
Approach
Treat a single trailing whitespace/punctuation token in the chat template as a grey-box perturbation; measure ASR across eight open models, search the vocabulary with GCG at suffix length 1, and explain the effect via tokenization statistics of spaces in C4 and a list-generation shift, plus a small attention-based proposition.
Key results
- Appending a space raises Vicuna-7B ASR from 3.0 to 100.0 and Guanaco-7B to 100.0, while Llama-2 and Llama-3 stay near 0-3%.
- LoRA fine-tuning Vicuna-7B on LIMA with a prepended space reduces ASR to 23.0 versus 99.0 without it.
- Mistral-7B GSM8K exact-match (strict) drops only from 38.67 to 36.32 with space appended.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Append a space to the end of the default chat template | Open-source chat models · AdvBench harmful behaviors (100-sample subset) | ASR: Vicuna-7B 3.0 -> 100.0; Guanaco-7B 36.0 -> 100.0; Llama2-7B 0.0 -> 0.0; Falcon-7B 73.0 -> 84.0; Mistral-7B 21.0 -> 58.0; | Mixed |
| Append a space to the end of the template | Mistral-7B · GSM8K | exact match: 38.67 -> 36.32 (strict-match); 42.15 -> 40.86 (flexible-extract) | Hurt |
| Fine-tune on LIMA samples with a space prepended (ablation) | Vicuna-7B · AdvBench harmful behaviors | ASR: 99.0 -> 23.0 | Helped |
| Move the space from the end to the start of the template (ablation) | Vicuna-7B · AdvBench harmful behaviors | ASR: 100.0 -> 3.0 | Helped |
| Move the space from the end of the template to before the user input (ablation) | Vicuna-7B · AdvBench harmful behaviors | ASR: 100.0 -> 3.0 | Helped |
| Move the space from the end of the template to after the user input (ablation) | Vicuna-7B · AdvBench harmful behaviors | ASR: 100.0 -> 3.0 | Helped |
| Replace single-space separators with double-space separators (ablation) | Vicuna-7B · AdvBench harmful behaviors | ASR: 3.0 -> 3.0 | No clear effect |
| Replace single-space separators with newline separators (ablation) | Vicuna-7B · AdvBench harmful behaviors | ASR: 3.0 -> 1.0 | Helped |
| Append punctuation or other single-character tokens to the template | 7B models · AdvBench harmful behaviors | ASR | Mixed |
| Append a single space to the end of the template | Vicuna-7B · AdvBench harmful behaviors | ASR: 3.0 -> 100.0 | Hurt |
Surprising findings
- Most single-character tokens, including many non-space punctuation marks and non-Latin characters, also achieve high ASR across models.
- Using a newline as a separator reduces harmful generations below Vicuna's base rate.
Limitations
- Only English inputs were examined.
- ASR relies on refusal keyword matching; manual check of 80 outputs found 6 non-harmful.
- Closed-source models could not be tested since template access is required.
Experimental setup
Datasets: AdvBench C4 LIMA GSM8K ShareGPT
Benchmarks: AdvBench GSM8K
Metrics: attack success rate (refusal keyword matching) base rate top-k overlap exact match
Baselines: GCG
Builds on: GCG RLHF Universal adversarial suffixes
Closest papers by meaning
- Safety Alignment Should be Made More Than Just a Few Tokens Deep (ICLR 2025 · similarity 0.90)
- Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space (NeurIPS 2024 · similarity 0.89)
- Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates (NeurIPS 2024 · similarity 0.89)
- Mission Impossible: A Statistical Perspective on Jailbreaking LLMs (NeurIPS 2024 · similarity 0.89)
- Fundamental Limitations of Alignment in Large Language Models (ICML 2024 · similarity 0.89)
- Inference-Time Vulnerability Beyond Shallow Safety: Alignment Along Generation Trajectories (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.