AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

GlitchCleaner: Lightweight Glitch Tokens Repairing by Lossless Gated LoRA in Large Language Models

Yibo Fan, Jingru Li, Huan Li

AAAI 2026 · Large language models › Reasoning and chain of thought · approach: Efficient LLM serving

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes GlitchCleaner, a lightweight repair method for glitch tokens in LLM vocabularies using lossless gated LoRA.

Problem

Glitch tokens in LLM vocabularies can trigger hallucinations and unpredictable or harmful outputs; detection methods exist but effective repair remains a challenge.

Approach

Repairs glitch tokens with a lossless gated LoRA module, aiming to fix problematic tokens while leaving the rest of the model's behavior intact (based on the title; abstract is truncated).

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace GlitchProber activation correction with gated LoRA branches trained on glitch tokensglitch token repair · Repair rate, 5 models (repeat task)repair rate: 50.08% -> 86.88%Improved
use gated LoRA instead of GlitchProbermodel general capability · GSM8K averagescore: 24.96 -> 31.04 (original 31.43)Improved
use gated LoRA instead of full fine-tuning on glitch tokensmodel general capability · MMLU averagescore: 49.03 -> 51.06Improved
add GlitchCleaner to newer modelsLlama-3.1-8B, Qwen3-8B, Mistral-v0.3 · GSM8K, MMLU, C-Eval, MetaBench, PIQA, AGIEvalscore: GSM8K 88.40 -> 87.83 (Qwen3-8B)No clear effect
use GlitchCleaner instead of GlitchProber for inferenceLlama-2-7b-chat · inference speed H200tokens/sec: 11.82 -> 62.83 (original 66.30)Improved
apply to spelling and length glitch typesGlitchCleaner · Llama-2-7b-chatrepair rate: 94.43% (spelling), 92.45% (length)Improved

Experimental setup

Builds on: LoRA

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy