Research map / Label-efficient and robust learning / Knowledge distillation
VRM: Knowledge Distillation via Virtual Relation Matching
Weijia Zhang, Fei Xie, Weidong Cai, Chao Ma
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace vanilla KD loss with VRM virtual relation matching | DeiT-T student with ResNet56 teacher · CIFAR-100 cross-architecture | top-1 accuracy: 65.08 -> 79.52 | Improved |
| use VRM relation matching instead of DIST on same-model KD | ResNet32x4 -> ResNet8x4 · CIFAR-100 | top-1 accuracy: 76.31 -> 78.76 | Improved |
| supervise student virtual-view predictions with ground-truth CE (ablation) | VRM · CIFAR-100 ResNet32x4 -> ResNet8x4 | top-1 accuracy: 78.47 -> 78.76 | Improved |
| supervise student virtual-view predictions with ground-truth CE (ablation) | VRM · CIFAR-100 VGG13 -> VGG8 | top-1 accuracy: 75.66 -> 76.19 | Improved |
| add vertex matching to edge matching (ablation) | VRM · CIFAR-100 | top-1 accuracy | Hurt |
| train for 360 epochs instead of 240 (ablation) | VRM · CIFAR-100 | top-1 accuracy: +0.21% | Improved |
| use class-wise pairwise-difference relations instead of Gram matrices or angle-wise relations (ablation) | VRM · CIFAR-100 | top-1 accuracy | Improved |
| prune redundant edges (REP) from the augmented graph (ablation) | VRM · CIFAR-100 | top-1 accuracy and efficiency | Improved |
| prune unreliable edges (UEP) by joint entropy (ablation) | VRM · CIFAR-100 | top-1 accuracy | Improved |
| use weak-only transformations for both real and virtual views (ablation) | VRM · CIFAR-100 / ImageNet | top-1 accuracy | No clear effect |
| increase batch size (ablation) | DIST relation matching versus VRM · CIFAR-100 | top-1 accuracy | Mixed |
| apply VRM to object detection versus feature-based FCFD | Faster-RCNN ResNet18 student · MS-COCO | AP: 37.37 -> 35.46 (FCFD vs VRM) | Hurt |
Closest papers by meaning
- Complementary Relation Contrastive Distillation (CVPR 2021 · similarity 0.89)
- Relational Knowledge Distillation (CVPR 2019 · similarity 0.89)
- Show, Attend and Distill: Knowledge Distillation via Attention-based Feature Matching (AAAI 2021 · similarity 0.89)
- Rethinking the Dark Knowledge and Kullback-Leibler Divergence Loss in Knowledge Distillation Under Capacity Mismatching (AAAI 2026 · similarity 0.88)
- CIFD: Controlled Information Flow to Enhance Knowledge Distillation (NeurIPS 2024 · similarity 0.87)
- Single-Teacher View Augmentation: Boosting Knowledge Distillation via Angular Diversity (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
