Research map / Optimization / Implicit bias and generalization
Sign Gradient Descent-based Neuronal Dynamics: ANN-to-SNN Conversion Beyond ReLU Network
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that discrete integrate-and-fire dynamics equal a subgradient method and uses a sign-gradient-descent neuron to convert ANNs with diverse nonlinearities (ConvNext, MLP-Mixer, ResMLP) to SNNs at low time steps.
Problem
Discrete neuronal dynamics of spiking neurons lack theory, limiting ANN-to-SNN conversion accuracy at low time steps and restricting support to ReLU-only networks.
Approach
Proves IF neuron dynamics with rate-coded input equal a subgradient method over an unconstrained convex problem, then replaces the subgradient update with sign gradient descent (signGD) and a generalized learning-rate schedule to encode a binary spike train that approximates diverse nonlinearities beyond ReLU.
Key results
- ImageNet top-1 of converted VGG16 and ResNet34 exceeds 75% at T=64, outperforming runner-up conversion methods by about 3%.
- ConvNext-B and RegNetX-3.2F reach roughly 81% top-1 at T=256, the first conversion of these architectures.
- ResNet18 on CIFAR-10 converted with the signGD neuron reaches 94% accuracy in 22 steps versus 64 steps for the subgradient-based neuron; signGD neuron costs about 1.6x theoretical energy per timestep.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use signGD-based neurons instead of subgradient-based neurons | ResNet18 converted SNN · CIFAR10 | time steps at 94% accuracy: 64 -> 22 | Helped |
| use signGD-based neurons instead of IF / LIF neurons | converted SNN inference · CIFAR10, ResNet18 and VGG16, T = 64 | energy: 20.188 -> 32.298; 20.386 -> 32.298; 9.700 -> 16.006; 9.893 -> 16.006 | Hurt |
| use signGD-based neurons instead of IF / LIF neurons | single synaptic operation · 45nm CMOS | energy per SOP: 0.9pJ -> 1.8pJ | Hurt |
Surprising findings
- Theoretical energy per spike operation is 1.8 pJ for the signGD neuron versus 0.9 pJ for IF/LIF, so per-timestep energy rises about 1.6x despite fewer time steps needed.
Limitations
- Layer normalization and square/inverse-sqrt approximations converge slowly, so non-ReLU architectures like MLP-Mixer and ConvNext remain slow to convert.
- Energy analysis is theoretical only; no hardware implementation or empirical measurement.
- Signed neuron requires two or more internal variables, making it more complex than LIF for neuromorphic hardware.
Experimental setup
Datasets: ImageNet CIFAR-10 CIFAR-100
Benchmarks: ImageNet CIFAR-10 CIFAR-100
Metrics: top-1 accuracy energy consumption (theoretical, uJ) firing rate
Baselines: QCFS StepReLU SlipReLU subgradient-based IF neuron SpikingYOLO
Builds on: QCFS subgradient method signGD
Closest papers by meaning
- NDOT: Neuronal Dynamics-based Online Training for Spiking Neural Networks (ICML 2024 · similarity 0.90)
- Inference-Scale Complexity in ANN-SNN Conversion for High-Performance and Low-Power Applications (CVPR 2025 · similarity 0.90)
- EnOF-SNN: Training Accurate Spiking Neural Networks via Enhancing the Output Feature (NeurIPS 2024 · similarity 0.90)
- Temporal Misalignment in ANN-SNN Conversion and its Mitigation via Probabilistic Spiking Neurons (ICML 2025 · similarity 0.89)
- Faster and Stronger: When ANN-SNN Conversion Meets Parallel Spiking Calculation (ICML 2025 · similarity 0.89)
- Adaptive Calibration: A Unified Conversion Framework of Spiking Neural Networks (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.