Research map / Architectures and efficiency / Spiking networks
EnOF-SNN: Training Accurate Spiking Neural Networks via Enhancing the Output Feature
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EnOF-SNN, which improves spiking neural network training by aligning SNN output features through the frozen ANN classifier (LAF loss, a KL-based distillation) and replacing the last LIF layer with ReLU (RepAct) to obtain full-precision output features.
Problem
Spiking neural networks suffer accuracy degradation relative to ANNs because binary spike feature maps limit the expressiveness of the output feature (the representation before the classifier).
Approach
Feed both SNN and ANN output features through the same pre-trained ANN classifier and align the resulting softmax outputs with a KL-divergence loss (LAF), and replace the final LIF activation with ReLU (RepAct) so the output feature is full precision.
Key results
- ResNet19 on CIFAR-10 reaches 96.19% top-1 with 2 timesteps, versus 95.53% for RecDis-SNN at 4 timesteps.
- ResNet19 on CIFAR-100 reaches 82.43% top-1 with 2 timesteps, outperforming TET and TEBN at 6 timesteps by 7.71% and 6.02%.
- On ImageNet, ResNet34 reaches 67.40% top-1 with 4 timesteps, and on CIFAR10-DVS ResNet20 reaches 80.50%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add LAF loss (ablation) | ResNet20 SNN · CIFAR-10, timestep 1 | accuracy: 90.40% -> 92.28% | Helped |
| Replace the last LIF activation with ReLU (RepAct) (ablation) | ResNet20 SNN · CIFAR-10, timestep 1 | accuracy: 90.40% -> 92.50% | Helped |
| Add LAF loss and replace the last LIF activation with ReLU (ablation) | ResNet20 SNN · CIFAR-10, timestep 1 | accuracy: 90.40% -> 92.66% | Helped |
| Add LAF loss (ablation) | ResNet20 SNN · CIFAR-10, timestep 2 | accuracy: 92.80% -> 93.53% | Helped |
| Replace the last LIF activation with ReLU (RepAct) (ablation) | ResNet20 SNN · CIFAR-10, timestep 2 | accuracy: 92.80% -> 93.62% | Helped |
| Add LAF loss and replace the last LIF activation with ReLU (ablation) | ResNet20 SNN · CIFAR-10, timestep 2 | accuracy: 92.80% -> 93.86% | Helped |
| Add LAF loss (ablation) | ResNet20 SNN · CIFAR-10, timestep 4 | accuracy: 93.85% -> 94.39% | Helped |
| Replace the last LIF activation with ReLU (RepAct) (ablation) | ResNet20 SNN · CIFAR-10, timestep 4 | accuracy: 93.85% -> 94.66% | Helped |
| Add LAF loss and replace the last LIF activation with ReLU (ablation) | ResNet20 SNN · CIFAR-10, timestep 4 | accuracy: 93.85% -> 94.74% | Helped |
| Use LAF + RepAct | ResNet19 SNN · CIFAR-10 | top-1 accuracy: 95.55% -> 96.19%±0.10 | Helped |
| Use LAF + RepAct | ResNet19 SNN · CIFAR-100 | top-1 accuracy: 76.41% -> 82.43%±0.09 | Helped |
| Use LAF + RepAct | ResNet34 SNN · ImageNet | top-1 accuracy: 67.74% -> 67.40%±0.10 | Hurt |
Surprising findings
- Replacing only the last LIF layer with ReLU (RepAct) alone yields a gain of over 2 points at timestep 1 on CIFAR-10 ResNet20, comparable to LAF alone.
Limitations
- Last layer uses full-precision ReLU rather than binary spikes, so the output is not fully spike-based and some multiplications remain.
- Results are reported with the single-hyperparameter lambda tuned on CIFAR-10 (lambda=0.1), with lambda=1 used on ImageNet.
- Requires a pre-trained ANN counterpart with the same architecture to supply the classifier, adding training cost.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet CIFAR10-DVS
Benchmarks: CIFAR-10 CIFAR-100 ImageNet CIFAR10-DVS
Metrics: top-1 accuracy
Baselines: RecDis-SNN TET STBP-tdBN Diet-SNN TEBN GLIF Real Spike Ternary Spike Dspike
Builds on: STBP tdBN knowledge distillation RecDis-SNN
Closest papers by meaning
- Efficient ANN-Guided Distillation: Aligning Rate-based Features of Spiking Neural Networks through Hybrid Block-wise Replacement (CVPR 2025 · similarity 0.92)
- Enhanced Self-Distillation Framework for Efficient Spiking Neural Network Training (NeurIPS 2025 · similarity 0.91)
- Synergy Between the Strong and the Weak: Spiking Neural Networks are Inherently Self-Distillers (NeurIPS 2025 · similarity 0.91)
- S$^2$NN: Sub-bit Spiking Neural Networks (NeurIPS 2025 · similarity 0.90)
- Enhancing Representation of Spiking Neural Networks via Similarity-Sensitive Contrastive Learning (AAAI 2024 · similarity 0.90)
- Rethinking the Membrane Dynamics and Optimization Objectives of Spiking Neural Networks (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.