Research map / Large language models / Reasoning and chain of thought
GlitchCleaner: Lightweight Glitch Tokens Repairing by Lossless Gated LoRA in Large Language Models
Yibo Fan, Jingru Li, Huan Li
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GlitchCleaner, a lightweight repair method for glitch tokens in LLM vocabularies using lossless gated LoRA.
Problem
Glitch tokens in LLM vocabularies can trigger hallucinations and unpredictable or harmful outputs; detection methods exist but effective repair remains a challenge.
Approach
Repairs glitch tokens with a lossless gated LoRA module, aiming to fix problematic tokens while leaving the rest of the model's behavior intact (based on the title; abstract is truncated).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace GlitchProber activation correction with gated LoRA branches trained on glitch tokens | glitch token repair · Repair rate, 5 models (repeat task) | repair rate: 50.08% -> 86.88% | Improved |
| use gated LoRA instead of GlitchProber | model general capability · GSM8K average | score: 24.96 -> 31.04 (original 31.43) | Improved |
| use gated LoRA instead of full fine-tuning on glitch tokens | model general capability · MMLU average | score: 49.03 -> 51.06 | Improved |
| add GlitchCleaner to newer models | Llama-3.1-8B, Qwen3-8B, Mistral-v0.3 · GSM8K, MMLU, C-Eval, MetaBench, PIQA, AGIEval | score: GSM8K 88.40 -> 87.83 (Qwen3-8B) | No clear effect |
| use GlitchCleaner instead of GlitchProber for inference | Llama-2-7b-chat · inference speed H200 | tokens/sec: 11.82 -> 62.83 (original 66.30) | Improved |
| apply to spelling and length glitch types | GlitchCleaner · Llama-2-7b-chat | repair rate: 94.43% (spelling), 92.45% (length) | Improved |
Experimental setup
Builds on: LoRA
Closest papers by meaning
- Scalable Token-Level Hallucination Detection in Large Language Models (NeurIPS 2026 · similarity 0.87)
- Token-Guard: Towards Token-Level Hallucination Control via Self-Checking Decoding (ICLR 2026 · similarity 0.87)
- BALTO: Balanced Token-Level Policy Optimization for Hallucination Mitigation (NeurIPS 2026 · similarity 0.86)
- Fail-Closed Alignment for Large Language Models (NeurIPS 2026 · similarity 0.86)
- LLM-Check: Investigating Detection of Hallucinations in Large Language Models (NeurIPS 2024 · similarity 0.86)
- Generate, but Verify: Reducing Hallucination in Vision-Language Models with Retrospective Resampling (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
