AI Paper Atlas

Research map / Label-efficient and robust learning / Knowledge distillation

EM-KD: Distilling Efficient Multimodal Large Language Model with Unbalanced Vision Tokens

Ze Feng, Sen Yang, Boqiang Duan, Wankou Yang, Jingdong Wang

AAAI 2026 · Label-efficient and robust learning › Knowledge distillation · approach: Few-shot and metric distillation · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add vision-language affinity distillation (VLAD) to the SFT baseline (ablation)0.6B efficient student · 11-benchmark averageaverage accuracy: 47.7 -> 48.4Improved
add vision semantic distillation (VSD) on top of VLAD (ablation)0.6B efficient student · 11-benchmark averageaverage accuracy: 48.4 -> 49.5Improved
add response logits distillation (RLD) on top of VLAD and VSD (ablation)0.6B efficient student · 11-benchmark averageaverage accuracy: 49.5 -> 50.4Improved
add VLAD alone (ablation)0.6B efficient student · Knowledge benchmarks averageaccuracy: 42.4 -> 41.1Hurt
match tokens with Hungarian algorithm on vision logits instead of average pooling (ablation)EM-KD · 11-benchmark averageaverage accuracy: 48.6 -> 50.4Improved
match with vision logits instead of hidden states in Hungarian matching (ablation)EM-KD · 11-benchmark averageaverage accuracy: 49.4 -> 50.4Improved
distill vision logits instead of hidden states in VSD (ablation)EM-KD · 11-benchmark averageaverage accuracy: 49.5 -> 50.4Improved
use EM-KD instead of LLaVA-KD with Vision Token Matching0.6B efficient student · 11-benchmark averageaverage accuracy: 49.7 -> 50.4Improved
use EM-KD instead of MiniLLM response-only distillation0.6B efficient student · 11-benchmark averageaverage accuracy: 49.5 -> 50.4Improved
distill from LLaVA-OneVision-SI-7B teacher into 8B efficient student with EM-KD8B efficient student · 11-benchmark averageaverage accuracy: 61.7 -> 63.4Improved
train student with EM-KD instead of training-free pruning or compression at 144 tokensLLaVA-NeXT 0.5B · RealWorldQA TTFTTTFT (ms): 103.3 -> 54.9Improved

Later papers that compared with this method

1 later accepted paper reports EMKD as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy