Research map / Label-efficient and robust learning / Knowledge distillation
SFKD: Spatial–Frequency Joint-Aware Heterogeneous Knowledge Distillation via Multi-Level Wavelet Spectral Interaction
Cuipeng Wang, Haipeng Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SFKD instead of FBT | heterogeneous KD · CIFAR-100 heterogeneous, average | accuracy gain: 8.38 -> 10.92 | Improved |
| use SFKD instead of FBT | heterogeneous KD · ImageNet-1K heterogeneous, average | accuracy gain: 2.31 -> 2.51 | Improved |
| use SFKD instead of FBT | heterogeneous KD · CIFAR-100 heterogeneous, ViT-S -> MobileNetV2 | top-1 accuracy: 82.10 -> 79.97 | Hurt |
| use SFKD instead of FitNet feature projection | Transformer student · CIFAR-100 heterogeneous, ConvNeXt-T -> Swin-P | top-1 accuracy: 24.06 -> 82.16 | Improved |
| use SFKD instead of FBT | homogeneous KD · ImageNet-1K homogeneous, ResNet50 -> MobileNet | top-1 accuracy: 73.45 -> 73.81 | Improved |
| add L1 low-frequency loss (ablation) | SFKD · ImageNet-1K ablation, Swin-T -> ResNet18 | top-1 accuracy: 72.19 -> 72.54 | Improved |
| add L2 high-frequency loss (ablation) | SFKD · ImageNet-1K ablation, Swin-T -> ResNet18 | top-1 accuracy: 72.32 -> 72.54 | Improved |
| add L3 reconstruction loss (ablation) | SFKD · ImageNet-1K ablation, Swin-T -> ResNet18 | top-1 accuracy: 72.31 -> 72.54 | Improved |
| add L4 KL loss (ablation) | SFKD · ImageNet-1K ablation, Swin-T -> ResNet18 | top-1 accuracy: 72.46 -> 72.54 | Improved |
| add LCR refinement stream (ablation) | SFKD · ImageNet-1K ablation, Swin-T -> ResNet18 (refinement streams) | top-1 accuracy: 71.24 -> 72.19 | Improved |
| add GTR refinement stream (ablation) | SFKD · ImageNet-1K ablation, Swin-T -> ResNet18 (refinement streams) | top-1 accuracy: 71.24 -> 72.46 | Improved |
| combine LCR and GTR streams (ablation) | SFKD · ImageNet-1K ablation, Swin-T -> ResNet18 (refinement streams) | top-1 accuracy: 72.46 -> 72.54 | Improved |
Closest papers by meaning
- FreeKD: Knowledge Distillation via Semantic Frequency Prompt (CVPR 2024 · similarity 0.87)
- Distilling Knowledge from Heterogeneous Architectures for Semantic Segmentation (AAAI 2025 · similarity 0.84)
- Spectral-Spatial Interpretation (NeurIPS 2026 · similarity 0.83)
- U-Know-DiffPAN: An Uncertainty-aware Knowledge Distillation Diffusion Framework with Details Enhancement for PAN-Sharpening (CVPR 2025 · similarity 0.83)
- Localizing Knowledge in Diffusion Transformers (NeurIPS 2025 · similarity 0.83)
- DCSF-KD: Dynamic Channel-wise Spatial Feature Knowledge Distillation for Object Detection (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
