AI Paper Atlas

Research map / Label-efficient and robust learning / Knowledge distillation

CMKD: CNN/Transformer-Based Cross-Model Knowledge Distillation for Audio Classification

Yuan Gong, Sameer Khurana, Andrew Rouditchenko, James Robert Glass

TPAMI 2025 · Label-efficient and robust learning › Knowledge distillation · approach: Few-shot and metric distillation · 41 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use CNN teacher instead of same-architecture teacher (cross-model vs self-KD) (ablation)AST-Base · FSD50K evalmAP: 59.6 -> 61.7Improved
use a smaller AST-Tiny teacher instead of the strongest AST-Base teacher (ablation)EfficientNet-B2 · FSD50K evalmAP: 57.3 -> 58.2Improved
distill from KD-trained teacher (iterative KD) instead of no-KD teacher (ablation)EfficientNet-B2 / AST-Base · FSD50K evalmAP: 58.2 -> 58.0 and 61.7 -> 61.4Hurt
raise temperature from 1.0 to 2.5 (ablation)EfficientNet-B2 (AST to CNN KD) · FSD50K evalmAP: 57.2 -> 58.2Improved
lower temperature from 1.0 to 0.5 (ablation)AST-Base (CNN to AST KD) · FSD50K evalmAP: 61.7 -> 60.9Hurt
add ground-truth loss to teacher soft labels (lambda 0 -> 0.5) (ablation)AST-Base · FSD50K evalmAP: 59.7 -> 61.7Improved
use consistent teacher (same augmented input as student) instead of conventional teacher (ablation)AST-Base (CNN to AST KD) · FSD50K evalmAP: 60.4 -> 61.7Improved
use consistent teacher instead of conventional teacher (ablation)EfficientNet-B2 (AST to CNN KD) · FSD50K evalmAP: 58.2 -> 58.2No clear effect
add a separate distillation head (ablation)AST-Base · FSD50K evalmAP: 61.7 -> 61.4Hurt
add a separate distillation head (ablation)EfficientNet-B2 · FSD50K evalmAP: 58.2 -> 56.9Hurt
add a second diverse teacher (EfficientNet-B0 + DenseNet-121) with multi-loss KDAST-Base · FSD50K evalmAP: 61.7 -> 62.3Improved
add multi-teacher KDEfficientNet-B2 · FSD50K evalmAP: 58.2 -> 58.0No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy