AI Paper Atlas

Research map / Label-efficient and robust learning / Knowledge distillation

Aggregating Cross-Domain Knowledge via Learnable Tokens for Multi-Teacher Distillation

Wu Ran, Weijia Zhang, ShuYang Pang, JingSheng Liu, Xiaohui Zhang, Yichao Yan, Chao Ma

ECCV 2026 · Label-efficient and robust learning › Knowledge distillation · approach: Few-shot and metric distillation · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add the learnable token (ablation)ACTok ConvNeXt-B student · patch alignment lossLf (lower better): 0.29 -> 0.23Improved
add the cross-space module (ablation)ACTok · ConvNeXt-B distillationpatch alignment loss: 0.35 -> 0.23Improved
add the intra-space module (ablation)ACTok · ConvNeXt-B distillationalignment losses: Lf 2.60 -> 0.23Improved
use cosine instead of MSE for DINO objective (ablation)ACTok ViT-L · fine-grained avgaccuracy: 90.9 -> 92.4Improved
scale lr of interaction modules by 0.1x during downstream tuning (ablation)ACTok ViT-L · CUBaccuracy: 89.2 -> 90.8Improved
use learnable-token head features instead of backbone (ablation)ACTok ResNet-50 · CUB2011accuracy: 73.8 -> 79.7 (CLS head)Improved
ViT teachers instead of CNN teachers for ConvNeXt-B studentACTok · ImageNet-1Ktop-1 accuracy: 83.0 -> 84.1Improved
distill from MAE/DINO CNN teachers into ConvNeXt-B studentACTok ConvNeXt-B · ImageNet-1Ktop-1 accuracy: 83.8 -> 83.0 vs supervisedHurt
distill into ResNet-50ACTok ResNet-50 · ADE20K linear probingmIoU: 12.4 -> 20.5Improved
distill into ResNet-50ACTok ResNet-50 · ImageNet-1Ktop-1 accuracy: 80.9 -> 78.0Hurt
ACTok ViT-B vs MAE teacherACTok ViT-B · ADE20K linear probingmIoU: 22.2 -> 46.5Improved
ACTok ViT-B vs DINOv3 teacherACTok ViT-B · ADE20K linear probingmIoU: 50.9 -> 46.5Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy