Research map / Large language models / Language, safety and interpretability
Understanding the Ability of LLMs to Handle Character-Level Perturbation
Anyuan Zhuo, Xuefei Ning, Ningyuan Li, Jingyi Zhu, Yu Wang, Pinyan Lu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| character-level tokenization of clean text (ablation) | Qwen3-30B-A3B · GSM8K | accuracy: 0.941 -> 0.914 | No clear effect |
| character-level tokenization of clean text (ablation) | Llama3.1-70B · GSM8K | accuracy: 0.936 -> 0.883 | Hurt |
| random in-word segmentation of clean text (ablation) | Qwen3-30B-A3B · GSM8K | accuracy: 0.941 -> 0.935 | No clear effect |
| use larger Qwen3 model (32B vs 30B-A3B) (ablation) | Qwen3 under 1-Inj · GSM8K | accuracy: 43.4% -> 53.6% | Improved |
| switch from Qwen3 to Llama3.1-70B (ablation) | models under 1-Inj · GSM8K | accuracy: 43.4% -> 10.0% | Hurt |
| add longer irrelevant context prefix (ablation) | Qwen3-30B-A3B under Typo and Shuffle · GSM8K | accuracy: 0.482 -> 0.614 (30-Typo, prefix2) | Improved |
| add longer irrelevant context prefix (ablation) | Qwen3-30B-A3B under UCC-Inj · GSM8K | accuracy: 0.434 -> 0.052 (1-Inj, prefix3) | Hurt |
| explicitly instruct model to remove noisy characters (ablation) | Qwen3-30B-A3B under 1-Inj · GSM8K | accuracy: 0.434 -> 0.660 | Improved |
| add 1-shot example of explicit rewriting (ablation) | Qwen3-30B-A3B under 1-Inj · GSM8K | accuracy: 0.434 -> 0.788 | Improved |
| force explicit problem rewriting in CoT under weak perturbation (ablation) | Qwen3-30B-A3B under 30-Shuffle · GSM8K | accuracy: 0.716 -> 0.760 | Improved |
| force explicit rewriting under strong perturbation (ablation) | Qwen3-30B-A3B under 100-Shuffle · GSM8K | accuracy: 0.407 -> 0.345 | Hurt |
| sample VS characters from full 256-set instead of 1 (ablation) | UCC-Inj on Qwen3-30B-A3B · GSM8K | accuracy: 67.1% -> 2.2% | Hurt |
Closest papers by meaning
- Remarkable Robustness of LLMs: Stages of Inference? (NeurIPS 2025 · similarity 0.90)
- What Makes Quantization for Large Language Model Hard? An Empirical Study from the Lens of Perturbation (AAAI 2024 · similarity 0.89)
- Brittlebench: Quantifying LLM robustness via prompt sensitivity (NeurIPS 2026 · similarity 0.89)
- When LLMs Meet Cunning Texts: A Fallacy Understanding Benchmark for Large Language Models (NeurIPS 2024 · similarity 0.88)
- SETLEXSEM CHALLENGE: Using Set Operations to Evaluate the Lexical and Semantic Robustness of Language Models (NeurIPS 2024 · similarity 0.88)
- From Tokens to Words: On the Inner Lexicon of LLMs (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
