Research map / Label-efficient and robust learning / Knowledge distillation
A Good Student is Cooperative and Reliable: CNN-Transformer Collaborative Learning for Semantic Segmentation
Jinjing Zhu, Yunhao Luo, Xu Zheng, Hao Wang, Lin Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add BSD (bidirectional selective distillation) (ablation) | CNN-ViT online KD · PASCAL VOC 2012 (MobileNetV2 + MiT-B1) | mIoU delta sum: 0 -> +2.08 | Improved |
| add HFD (heterogeneous feature distillation) (ablation) | CNN-ViT online KD · PASCAL VOC 2012 (MobileNetV2 + MiT-B1) | mIoU delta sum: 0 -> +0.65 | Improved |
| add HFD on top of BSD (ablation) | CNN-ViT online KD · PASCAL VOC 2012 (MobileNetV2 + MiT-B1) | mIoU delta sum: +2.08 -> +2.72 | Improved |
| add pixel-wise BSD over region-wise alone (ablation) | BSD · PASCAL VOC 2012 (MobileNetV2 + MiT-B1) | mIoU delta sum: +0.72 -> +2.08 | Improved |
| add region-wise BSD over pixel-wise alone (ablation) | BSD · PASCAL VOC 2012 (MobileNetV2 + MiT-B1) | mIoU delta sum: +1.66 -> +2.08 | Improved |
| apply isomorphic KD methods (IFVD) between CNN and ViT | online KD between CNN and ViT · PASCAL VOC 2012 (MobileNetV2 + MiT-B1) | mIoU delta sum: -0.71 | Hurt |
| increase HFD weight beta from 0.1 to 1.0 (ablation) | HFD · PASCAL VOC 2012 (MobileNetV2 + MiT-B1) | mIoU delta sum: +0.65 -> -0.15 | Hurt |
| increase alpha from 1.0 to 5.0 (ablation) | BSD · PASCAL VOC 2012 (MiT-B1 student) | mIoU: MobileNetV2 69.19 -> 70.69; MiT-B1 78.91 -> 78.25 | Mixed |
| increase gamma from 1.0 to 5.0 (ablation) | BSD weight · PASCAL VOC 2012 (MiT-B1 student) | mIoU: MobileNetV2 69.03 -> 70.12; MiT-B1 78.94 -> 78.30 | Mixed |
| apply proposed method instead of prior KD methods | CamVid students · CamVid | mIoU delta sum: +2.75 (MobileNetV2+MiT-B1) | Improved |
| distill from CNN to ViT | MiT-B1 student · PASCAL VOC 2012 | mIoU: 78.48 -> 79.17 | No clear effect |
Closest papers by meaning
- Distilling Knowledge from Heterogeneous Architectures for Semantic Segmentation (AAAI 2025 · similarity 0.90)
- Multi-Knowledge Aggregation and Transfer for Semantic Segmentation (AAAI 2022 · similarity 0.90)
- Structured Knowledge Distillation for Semantic Segmentation (CVPR 2019 · similarity 0.89)
- Cumulative Spatial Knowledge Distillation for Vision Transformers (ICCV 2023 · similarity 0.89)
- ScaleKD: Strong Vision Transformers Could Be Excellent Teachers (NeurIPS 2024 · similarity 0.87)
- Mutual Learning for SAM Adaptation: A Dual Collaborative Network Framework for Source-Free Domain Transfer (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
