AI Paper Atlas

Research map / Label-efficient and robust learning / Knowledge distillation

Bridging the Tokenizer Gap: Semantics and Distribution-aware Knowledge Transfer for Unbiased Cross-Tokenizer Distillation

Huazheng Wang, Yongcheng Jing, Haifeng Sun, Jingyu Wang, Jianxin Liao, Leszek Rutkowski, Dacheng Tao

AAAI 2026 · Label-efficient and robust learning › Knowledge distillation · approach: Rethinking distillation objectives · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use SEDI instead of the best cross-tokenizer baseline DSKDstudent distillation · Dolly (LLaMA2-7B -> GPT2-124M)ROUGE-L: 23.33 -> 24.38Improved
use SEDI instead of DSKD on unseen UnNIstudent distillation · UnNI (Dolly-Pythia-3B -> OPT-125M)ROUGE-L: 17.36 -> 18.51Improved
use SEDI instead of DSKD on HumanEvalPythia-160M student · HumanEvalPass@1: 56.10 -> 60.98Improved
use SEDI instead of DSKD on HumanEval ROUGE-LPythia-160M student · HumanEvalROUGE-L: 23.01 -> 22.93No clear effect
use SEDI instead of DSKD on ORCAGPT2-Medium-355M student · ORCAPass@1: 2.93 -> 3.51Improved
bipartite graph alignment instead of edit-distance alignmenttoken alignment · DollyTop-1 accuracy (%): 84.94 -> 95.27Improved
increase top-K from 10 to 100 (ablation)SEDI · S-NIROUGE-L: 14.97 -> 18.18Improved
increase top-K from 100 to 400 (ablation)SEDI · UnNIROUGE-L: 18.51 -> 18.57No clear effect
use teacher with larger vocabulary (DeepSeek-R1-Distill) vs MetaMath-7Bstudent distillation · MetaMath-trained studentsPass@1 vs ROUGE-L: GSM8K Pass@1 7.65 vs 3.94Mixed
add entropy alignment with student logits (SEDI)student distillation · S-NI/UnNI OPT-125MExAccErrImproved
use SEDI instead of baselines for fluencyLLaMA-GPT pair · DollyFluency: 4.81 -> 4.91No clear effect
use SEDI instead of DSKD training costLLaMA-GPT pair · Dollytraining time per step (s): 1.5276 -> 1.8432Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy