GlitchPatch: Repairing Glitch Tokens in Frozen Language Models via Local Retokenization
Glitch tokens are anomalous vocabulary entries that can cause large language models (LLMs) to produce outputs inconsistent with their inputs. Existing repair methods require access to model internals, making them impractical for frozen checkpoints.
arXiv - cs.CL (NLP)2 min read