AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Beyond Perplexity: UTF-8 Validity in Byte-aware Language Models

Sangwhan Moon, Daisuke Oba, Youmi Ma, Tatsuya Hiraoka, Naoaki Okazaki

ICML 2026 · Large language models › Language, safety and interpretability · approach: Pruning and circuits in LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
evaluate on unseen 4-byte characters instead of common characters355M byte-fallback LM · Level 0 partial-credit validitypartial-credit validity (%): 96.21 -> 86.97Hurt
use 3-byte unseen control instead of 4-byte unseen at prefix=1 (ablation)355M byte-fallback LM · Level 0 unseen charspartial-credit validity (%): 0.0 -> 48.2Improved
use 3-byte unseen control instead of 4-byte unseen at prefix=2 (ablation)355M byte-fallback LM · Level 0 unseen charspartial-credit validity (%): 39.5 -> 51.1Improved
use 4-byte unseen instead of 3-byte unseen at prefix=3 (ablation)355M byte-fallback LM · Level 0 unseen charspartial-credit validity (%): 19.4 -> 60.5Mixed
vary character frequency within the 3-byte class (ablation)355M byte-fallback LM · Level 0partial-credit validity: 0.878 to 0.894No clear effect
add semantic context (Level 1 vs Level 0)355M byte-fallback LM · Common tiervalidity convergence speedImproved
compare validity with perplexity convergence in Level 0355M byte-fallback LM · Level 0convergence steps: 740 steps vs 380 stepsMixed
scale from Llama-3.2 1B to 3BLlama-3.2 · Cross-modelstrict validity V_s (%): 38.4 -> 49.0Improved
scale from Qwen-3.5 4B to 9BQwen-3.5 · Cross-modelstrict validity V_s (%): 87.0 -> 89.8Improved
scale from OLMo-2 1B to 7BOLMo-2 · Cross-modelstrict validity V_s (%): 33.5 -> 28.0Hurt
scale from Qwen-3.5 4B to 9BQwen-3.5 · Cross-model Level 1term match rate (%): 2.4 -> 0.5Hurt
use byte-fallback SentencePiece tokenizer instead of larger-vocabulary BPELlama-2 7B / Mistral 7B · Cross-model Level 1term match rate (%): 33.1 vs 0.5Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy