Research map / Architectures and efficiency / Spiking networks
FEEL-SNN: Robust Spiking Neural Networks with Frequency Encoding and Evolutionary Leak Factor
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FEEL-SNN, combining frequency encoding (time-varying frequency masks that suppress noise) and a trainable evolutionary leak factor in LIF neurons to improve adversarial and noise robustness of spiking neural networks.
Problem
Spiking neural networks remain vulnerable to adversarial and common noise; existing robustness analyses of SNNs are largely empirical, and standard direct encoding and fixed membrane leak ignore noise removal and temporal leak variation.
Approach
Derives a unified robustness bound for SNNs (Theorem 1) showing that the input-perturbation term depends on the per-time-step encoding and leak factor, then uses a frequency encoding that applies time-varying low-pass masks (radius decreasing over time) and a learnable per-neuron, per-time-step leak factor to reduce that term.
Key results
- On CIFAR-10 VGG11 (T=4), FEEL raises PGD robust accuracy from 15.59% (vanilla) to 30.27% with clean accuracy 92.73% vs 92.64%.
- On CIFAR-100 VGG11 (T=8, eps=8/255), RAT+FEEL reaches 31.68% FGSM and 18.07% PGD versus RAT's 29.25% and 15.43% (with StoG combined variants).
- Inverse frequency encoding (IFE) drops clean accuracy to 64.81%, supporting that informative content is concentrated in low frequencies.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add FEEL to vanilla SNN training | VGG11 · CIFAR10, T=4, white-box PGD | accuracy: 15.59 -> 30.27 | Helped |
| Add FEEL to vanilla SNN training | VGG11 · CIFAR100, T=8, white-box PGD | accuracy: 4.48 -> 12.35 | Helped |
| Add FEEL to vanilla SNN training | ResNet19 · Tiny-ImageNet, T=4, white-box PGD | accuracy: 2.91 -> 4.53 | Helped |
| Add FEEL to RAT | VGG11 · CIFAR100, T=8, white-box PGD | accuracy: 8.87 -> 12.36 | Helped |
| Add Frequency Encoding without Evolutionary Leak factor (ablation) | Vanilla VGG11 · CIFAR10, T=4, white-box PGD | accuracy: 15.59 -> 21.56 | Helped |
| Reverse the frequency-cropping order from high-to-low to low-to-high (ablation) | Frequency Encoding on VGG11 · CIFAR10, T=4, white-box PGD | accuracy: 21.56 -> 4.44 | Hurt |
| Reverse the frequency-cropping order from high-to-low to low-to-high (ablation) | Frequency Encoding on VGG11 · CIFAR10, T=4, clean | accuracy: 92.26 -> 64.81 | Hurt |
| Use a fixed frequency-mask radius across time steps instead of a different radius per step (ablation) | Frequency Encoding on VGG11 · CIFAR10, T=4, white-box PGD | accuracy: 6.89 vs. 2.28; 5.41; 5.38; 4.74 | Hurt |
| Fix the leak factor at λ = 0.0 (ablation) | FEEL on VGG11 · CIFAR10, T=4, white-box PGD | accuracy: 15.59 -> 63.80 | Helped |
| Fix the leak factor at λ = 0.0 (ablation) | FEEL on VGG11 · CIFAR10, T=4, clean | accuracy: 92.64 -> 81.76 | Hurt |
| Add L2 regularization to the evolutionary leak factor (ablation) | FEEL on VGG11 · CIFAR10, T=4, white-box PGD | accuracy: 29.98 vs. 30.27 | No clear effect |
| Use Evolutionary Leak factor without Frequency Encoding (ablation) | Vanilla VGG11 · CIFAR100, T=8, white-box PGD | accuracy: 4.48 -> 12.95 | Helped |
Surprising findings
- Smaller fixed leak factors improve adversarial robustness but substantially reduce clean accuracy (e.g., 81.76% at lambda=0 vs 92.64% vanilla), motivating a learnable leak.
- Frequency encoding alone slightly lowers clean accuracy on some settings (e.g., Vanilla+FE 72.67 vs 72.93 on CIFAR100).
Limitations
- Evaluated only on static image datasets; encoding for event-based DVS data is left for future work.
- No error bars or multiple seeds reported; a shared random seed is used.
- Black-box attacks use a model trained with a different seed rather than a different architecture.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny-ImageNet
Benchmarks: CIFAR-10 CIFAR-100 Tiny-ImageNet
Metrics: clean accuracy robust accuracy under FGSM, PGD, BIM, CW, Gaussian noise att
Baselines: Vanilla SNN AT RAT StoG Inverse-FE Direct encoding
Builds on: LIF neuron BPTT / STBP adversarial training StoG RAT
Closest papers by meaning
- Robust Stable Spiking Neural Networks (ICML 2024 · similarity 0.91)
- Toward Robust Spiking Neural Network Against Adversarial Perturbation (NeurIPS 2022 · similarity 0.90)
- Enhancing the Robustness of Spiking Neural Networks with Stochastic Gating Mechanisms (AAAI 2024 · similarity 0.89)
- Enhancing Adversarial Robustness in SNNs with Sparse Gradients (ICML 2024 · similarity 0.88)
- Rethinking the Membrane Dynamics and Optimization Objectives of Spiking Neural Networks (NeurIPS 2024 · similarity 0.88)
- EnOF-SNN: Training Accurate Spiking Neural Networks via Enhancing the Output Feature (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.