Research map / Architectures and efficiency / Spiking networks
Enhancing Representation of Spiking Neural Networks via Similarity-Sensitive Contrastive Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a similarity-sensitive contrastive learning framework that trains spiking neural networks by maximizing mutual information with a well-trained ANN teacher at each layer, weighted by CKA similarity.
Problem
Binary spike representations limit the information capacity of SNNs, causing accuracy degradation relative to ANNs on static and neuromorphic image classification.
Approach
Extends ANN-to-SNN distillation beyond joint-distribution KL alignment by adding an NCE-based contrastive lower bound on mutual information (positive pairs from same input, negatives from different inputs) and weighting each layer's loss by the inverse of its CKA similarity to the ANN.
Key results
- Achieves 78.79% top-1 on CIFAR-100 with ResNet19 at 2 timesteps, and 76.37% with VGG16 at 5 timesteps.
- Reaches 66.78% top-1 on ImageNet with ResNet34 at 4 timesteps, below SEW ResNet (67.04%) and Real Spike (67.69%).
- Achieves 80.00% on CIFAR10-DVS with ResNet19 at 10 timesteps; ablation on CIFAR-100 ResNet20 shows 2-timestep accuracy rising from 68.27% to 69.81%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add distillation (ablation) | ResNet20 SNN · CIFAR-100 (timestep=2) | accuracy: 68.27% -> 68.74% | Helped |
| Add uniformly weighted contrastive loss (ablation) | ResNet20 SNN · CIFAR-100 (timestep=2) | accuracy: 68.74% -> 69.25% | Helped |
| Weight contrastive losses with CKA (SSCL rather than uniform weighting) (ablation) | ResNet20 SNN · CIFAR-100 (timestep=2) | accuracy: 69.25% -> 69.81% | Helped |
| Add distillation (ablation) | ResNet20 SNN · CIFAR-100 (timestep=4) | accuracy: 71.45% -> 71.72% | Helped |
| Add uniformly weighted contrastive loss (ablation) | ResNet20 SNN · CIFAR-100 (timestep=4) | accuracy: 71.72% -> 72.16% | Helped |
| Weight contrastive losses with CKA (SSCL rather than uniform weighting) (ablation) | ResNet20 SNN · CIFAR-100 (timestep=4) | accuracy: 72.16% -> 72.86% | Helped |
Surprising findings
- Uniformly weighted contrastive loss without CKA weighting already improves over the distillation baseline, and CKA weighting adds further gains (69.25% to 69.81% at 2 timesteps).
Limitations
- On ImageNet, the method trails SEW ResNet and Real Spike, which the authors attribute to those being non-standard SNN architectures.
- Requires first training a full ANN teacher with identical architecture, adding training cost.
- Accuracy gains are reported as mean over three runs; no energy measurements are given.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet CIFAR10-DVS
Benchmarks: CIFAR-10 CIFAR-100 ImageNet CIFAR10-DVS
Metrics: top-1 accuracy
Baselines: SpikeNorm Hybrid-Train STBP-tdBN TET GLIF Diet-SNN KDSNN SNNThroughKD TEBN Real Spike SEW ResNet RecDis-SNN
Builds on: Contrastive Multiview Coding (CMC) SNNThroughKD KDSNN STBP Centered kernel alignment
Closest papers by meaning
- EnOF-SNN: Training Accurate Spiking Neural Networks via Enhancing the Output Feature (NeurIPS 2024 · similarity 0.90)
- Self-cross Feature based Spiking Neural Networks for Efficient Few-shot Learning (ICML 2025 · similarity 0.88)
- S$^2$NN: Sub-bit Spiking Neural Networks (NeurIPS 2025 · similarity 0.86)
- Synergy Between the Strong and the Weak: Spiking Neural Networks are Inherently Self-Distillers (NeurIPS 2025 · similarity 0.86)
- Take A Shortcut Back: Mitigating the Gradient Vanishing for Training Spiking Neural Networks (NeurIPS 2024 · similarity 0.86)
- Efficient ANN-Guided Distillation: Aligning Rate-based Features of Spiking Neural Networks through Hybrid Block-wise Replacement (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.