Research map / Label-efficient and robust learning / Knowledge distillation
Bi-Spectrum Distillation: Addressing Spectral Mismatch in ANN-SNN Knowledge Transfer
Yuxuan Zhang, Yuhang Sun, Wen Yao, Yue Deng, Hongjue Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SRD (feature-level spectral residual) to logits KD (ablation) | SEW-ResNet-19 · CIFAR-100 | Top-1 accuracy (%): 79.46 -> 82.52 | Improved |
| add SSD (sorted-logit high-frequency distillation) to logits KD (ablation) | SEW-ResNet-19 · CIFAR-100 | Top-1 accuracy (%): 79.46 -> 79.98 | Improved |
| add SSD to logits KD (ablation) | SEW-ResNet-19 · CIFAR-10 | Top-1 accuracy (%): 96.33 -> 96.57 | Improved |
| add SRD to logits KD (ablation) | SEW-ResNet-19 · CIFAR-10 | Top-1 accuracy (%): 96.33 -> 96.53 | Improved |
| combine SRD and SSD (full BSD) versus SRD alone (ablation) | SEW-ResNet-19 · CIFAR-100 | Top-1 accuracy (%): 82.52 -> 82.61 | Improved |
| add standard logits KD over no distillation (ablation) | SEW-ResNet-19 · CIFAR-100 | Top-1 accuracy (%): 77.29 -> 79.46 | Improved |
| apply SRD to all three blocks instead of a single block (ablation) | ResNet-19 · CIFAR-10 | Top-1 accuracy (%): 95.87 -> 97.38 | Improved |
| use FFT-domain SRD instead of spatial-only adapter (ablation) | SEW-ResNet-18 · ImageNet | Top-1 accuracy (%): 65.47 -> 66.92 | Improved |
| apply BSD distillation on top of Sformer-12-512 initialized from E-SpikeFormer | Sformer-12-512 · ImageNet | Top-1 accuracy (%): 83.20 -> 85.33 | Improved |
| apply BSD on Sformer-4-384 versus BKDSNN | Sformer-4-384 · CIFAR-100 | Top-1 accuracy (%): 81.26 -> 83.51 | Improved |
| BSD versus TLD on ResNet-19 | ResNet-19 · CIFAR-100 | Top-1 accuracy (%): 82.47 -> 82.36 | No clear effect |
| SNN-to-SNN BSD distillation from T=16 teacher to T=10 student | Sformer-2-256 · CIFAR10-DVS | Top-1 accuracy (%): 79.80 -> 81.20 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- BKDSNN — BKDSNN: Enhancing the Performance of Learning-based Spiking Neural Networks Training with Blurred Knowledge Distillation (ECCV 2024) · better on 4 of 4 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Temporal Separation with Entropy Regularization for Knowledge Distillation in Spiking Neural Networks (CVPR 2025 · similarity 0.89)
- Efficient ANN-Guided Distillation: Aligning Rate-based Features of Spiking Neural Networks through Hybrid Block-wise Replacement (CVPR 2025 · similarity 0.89)
- Efficient Logit-based Knowledge Distillation of Deep Spiking Neural Networks for Full-Range Timestep Deployment (ICML 2025 · similarity 0.88)
- Synergy Between the Strong and the Weak: Spiking Neural Networks are Inherently Self-Distillers (NeurIPS 2025 · similarity 0.88)
- Enhanced Self-Distillation Framework for Efficient Spiking Neural Network Training (NeurIPS 2025 · similarity 0.88)
- EnOF-SNN: Training Accurate Spiking Neural Networks via Enhancing the Output Feature (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
