Research map / Architectures and efficiency / Spiking networks
Training High Performance Spiking Neural Network by Temporal Model Calibration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Temporal Model Calibration (TMC), a confidence-regularized cross-entropy loss that rescales temporal logit gradients across time steps so directly trained rate-coded SNNs get more diverse, temporally calibrated outputs and better accuracy.
Problem
Directly trained rate-coded SNNs have limited temporal heterogeneity because their temporal logit gradients are insufficiently diverse across time steps, leading to temporally miscalibrated, overconfident models with degraded performance.
Approach
Borrows the learning-based model calibration idea (confidence regularization on CE loss) and extends it to the temporal dimension: at each time step, add a regularizer on the target-class probability of the accumulated average logits (with stop-gradient on history), scaled by a base term and a linearly decreasing exponent, which rescales the logit gradient differently at each time step.
Key results
- On DVSCIFAR10 (VGG-SNN, T=10), TMC reaches 87.63% accuracy vs. 83.17% for TET and 73.86% for SDT.
- On ImageNet with Hierarchical SpikingTransformer (QKFormer, T=4), TMC reaches 85.83%, improving the initial QKFormer by 0.18%; with Meta-SpikeFormer it reaches 80.80%.
- On N-Caltech101 (VGG-SNN, T=16), TMC reaches 88.24%; at T=10 it reaches 86.03% vs. 81.72% for TET.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use TMC instead of TET | ResNet-19 · CIFAR100 | accuracy: 74.72 -> 78.05 | Helped |
| use TMC instead of TET | QKformer · ImageNet | accuracy: 85.66 -> 85.83 | Helped |
| use TMC instead of TET | VGG-SNN · DVSCIFAR10 | accuracy: 83.17 -> 87.63 | Helped |
| use TMC instead of TET | VGG-SNN · N-Caltech101 | accuracy: 81.72 -> 86.03 | Helped |
| replace θt with βt in the regularization term (ablation) | TMC on VGG-SNN · DVSCIFAR10 (T=10) | accuracy: 83.94 -> 81.82 | Hurt |
| add θt to the regularization term instead of βt (ablation) | TMC on VGG-SNN · DVSCIFAR10 (T=10) | accuracy: 81.82 -> 83.94 | Helped |
| remove λt from the regularization term (ablation) | TMC on VGG-SNN · DVSCIFAR10 (T=10) | accuracy: 83.94 -> 81.32 | Hurt |
| add λt to constrain regularization strength across time steps (ablation) | TMC on VGG-SNN · DVSCIFAR10 (T=10) | accuracy: 81.32 -> 83.94 | Helped |
Surprising findings
- On static CIFAR10/CIFAR100, TMC does not achieve the best results, trailing TEBN, RMP-Loss, TET, TKS, TCL, ETC, and TSSD, which the authors attribute to limited benefit of temporal heterogeneity on static data.
Limitations
- Gains on static datasets (CIFAR10/CIFAR100) are not competitive with several prior methods.
- For ImageNet QKFormer, the method is applied by fine-tuning a pretrained model for 10 epochs rather than training from scratch due to cost.
- Temporal calibration analysis relies on rate-coded SNN assumptions and the monotonic-confidence remark, which is argued rather than fully proven in the main text.
Experimental setup
Datasets: CIFAR10 CIFAR100 ImageNet DVSCIFAR10 N-Caltech101 DVS-Gesture SL-Animals-DVS Quora Question Pair
Benchmarks: CIFAR10 CIFAR100 ImageNet DVSCIFAR10 N-Caltech101 DVS-Gesture SL-Animals-DVS Quora Question Pair
Metrics: top-1 accuracy ECE AdaECE CECE
Baselines: SDT TET DSpike GLIF TEBN RMP-Loss TKS TCL ETC TSSD Spike-Driven Transformer V2 QKFormer
Builds on: BPTT with surrogate gradient Temporal Efficient Training Standard Direct Training label smoothing / confidence penalty calibration
Closest papers by meaning
- Temporal Flexibility in Spiking Neural Networks: Towards Generalization Across Time Steps and Deployment Friendliness (ICLR 2025 · similarity 0.91)
- TS-SNN: Temporal Shift Module for Spiking Neural Networks (ICML 2025 · similarity 0.91)
- Adaptive Calibration: A Unified Conversion Framework of Spiking Neural Networks (AAAI 2025 · similarity 0.90)
- Enhancing Training of Spiking Neural Network with Stochastic Latency (AAAI 2024 · similarity 0.90)
- Temporal Misalignment in ANN-SNN Conversion and its Mitigation via Probabilistic Spiking Neurons (ICML 2025 · similarity 0.90)
- Rethinking Spiking Neural Networks from an Ensemble Learning Perspective (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.