AI Paper Atlas

Research map / Large language models / Instruction and fine-tuning

TiTok: Transfer Token-level Knowledge via Contrastive Excess to Transplant LoRA

ChanJoo Jung, Jaehyung Kim

ICLR 2026 · Large language models › Instruction and fine-tuning · approach: Low-rank adaptation variants · 2 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add token-level excess-score token selection on top of random samples (ablation)TiTok · BBH (avg 4 transfers)accuracy: 0.458 -> 0.463Improved
add excess-score sample filtering (no token selection) (ablation)TiTok · BBH (avg 4 transfers)accuracy: 0.458 -> 0.470Improved
combine sample filtering and token selection (ablation)TiTok · BBH (avg 4 transfers)accuracy: 0.470 -> 0.483Improved
train on top 20% excess tokens instead of bottom 20% (ablation)TiTok Mistral 7B -> Llama3 8B · BBHaccuracy: 0.468 -> 0.482Improved
use all aligned token cases instead of one-to-one only (ablation)TiTok tokenizer alignment · BBH (Mistral 7B -> Llama3 8B)accuracy: 0.472 -> 0.482Improved
lower token ratio k% in weak-to-strong transfer (ablation)TiTok Llama2 7B -> Llama3 8B · BBHaccuracyImproved
lower token ratio k% in weak-to-strong transfer (ablation)TiTok Llama2 7B -> Llama3 8B · News HeadlineROUGE-LHurt
apply TiTok on external cross-task dataTiTok Mistral 7B -> Mistral 7B · News Headline cross-taskROUGE-L: 0.101 -> 0.120Improved
apply TiTok transfer to a 70B modelLlama-3.3-70B · BBH (4-bit)accuracy: 0.593 -> 0.621Improved
TiTok vs TransLoRA on MMLU in same-model transferTiTok Mistral 7B -> Mistral 7B · MMLUaccuracy: 0.534 -> 0.561Improved
TiTok vs vanilla on MMLU for Llama3 3B -> Llama3 8BTiTok · MMLUaccuracy: 0.469 -> 0.478Mixed
generate synthetic queries with source expert instead of target model (ablation)TiTok · BBHaccuracyImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy