Research map / Large language models / Instruction and fine-tuning

When the Same Coefficients Reach Different Places: Asymmetric Realizability in Transplanting Tokenizers across Large Language Models

NeurIPS 2026 · Large language models › Instruction and fine-tuning · approach: Efficient LLM fine-tuning systems · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that tokenizer transplant via shared-anchor coefficient reuse has an asymmetric realizability gap, letting an attacker plant a single donor-side "breaker" token that is inert in the donor but emits heavily in the downstream base.

Problem

Shared-basis (anchor-based) tokenizer transplant reuses donor-side coefficients on the base anchor dictionary; this can make donor-exclusive rows behave differently in base vs. donor, creating a supply-chain attack surface in open-weight model composition.

Approach

Identify asymmetric realizability: the same coefficient vector reaches different sets in donor and base anchor spans. Design a single appended donor token ("breaker") whose coefficients are low-energy in donor (via PCA-subspace penalty) yet reconstruct a high-salience base direction after transplant.

Key results

• On the 20-pair lightweight clique, 10/20 pairs reach SER_max >= 0.5 on the attacked base while keeping donor SER_max <= 0.05.
• One Gemma-2-2B donor compromises 13 downstream bases across five families; mean base SER exceeds 0.99 on 8 bases, while max donor SER over all cells is 0.03.
• Standard LoRA fine-tuning suppresses the breaker mainly on in-distribution prompts; far-OOD prompts retain SER above 0.87 across tested defenders.

Surprising findings

• Tested spectral filters (Z-scored residual norm, Magikarp) fail to detect breaker tokens, which sit in the dense center of the donor residual distribution.
• Random and high-norm one-token appends produce zero attacked-base SER, so the designed coefficients are necessary for the effect.

Limitations

• Evaluation limited to shared-basis training-free transplant operators (OMP, CLP, WECHSEL, FOCUS) and small-to-mid-size models.
• LoRA deployer case study limited to two base models and three SFT corpora; weight-merging result only in appendix.
• Authors withhold pre-compiled target-specific breaker artifacts; threat model assumes attacker knows public donor/base weights.

Experimental setup

Datasets: WikiText-103 Alpaca SQuAD v2 GSM8K

Benchmarks: WikiText-103 LAMBADA MMLU ARC-Challenge Alpaca SQuAD v2 GSM8K

Metrics: sequence emission rate perplexity accuracy breaker-token share output coherence

Baselines: random append high-norm append Magikarp

Builds on: mergekit-tokensurgeon Orthogonal Matching Pursuit CLP WECHSEL FOCUS

Closest papers by meaning

• Over-Tokenized Transformer: Vocabulary is Generally Worth Scaling (ICML 2025 · similarity 0.88)
• Model-Aware Tokenizer Transfer (NeurIPS 2026 · similarity 0.87)
• Explaining and Mitigating Crosslingual Tokenizer Inequities (NeurIPS 2025 · similarity 0.87)
• Enhancing Large Language Models through Adaptive Tokenizers (NeurIPS 2024 · similarity 0.86)
• Data Mixture Inference Attack: BPE Tokenizers Reveal Training Data Compositions (NeurIPS 2024 · similarity 0.86)
• Zero-Shot Tokenizer Transfer (NeurIPS 2024 · similarity 0.86)