Research map / Architectures and efficiency / Spiking networks
VISTREAM: Improving Computation Efficiency of Visual Streaming Perception via Law-of-Charge-Conservation Inspired Spiking Neural Network
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VISTREAM, which combines ST-BIF spiking neurons' Law of Charge Conservation with differential frame encoding so an SNN for visual streaming perception can skip per-frame membrane resets while matching its quantized-ANN counterpart.
Problem
Visual streaming perception on power-constrained edge devices is computationally expensive, and it is unproven whether SNNs offer accuracy and efficiency advantages over ANNs for VSP tasks, which also underuse temporal dependencies between frames.
Approach
Identifies the Law of Charge Conservation in ST-BIF neurons, which makes an SNN equivalent to a Q-ReLU ANN at equilibrium regardless of input timing; combined with differential encoding of neighboring frames (DiffEncode), the SNN keeps its membrane state across frames without resetting, reducing spike operations and energy.
Key results
- Var-IV (LoCC-SNN + DiffEncode) matches the ANN baseline's task performance exactly, since it is mathematically equivalent to Var-III (LoCC-SNN + VaniEncode).
- Relative to Var-III, Var-IV reduces operations by up to 46% and energy by up to 51%.
- IF-SNN with VaniEncode shows significant accuracy degradation versus the ANN on VSP tasks, and IF-SNN with DiffEncode fails completely at equilibrium.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add DiffEncode to LoCC-SNN (ablation) | LoCC-SNN · VSP tasks | energy: up to 51% | Helped |
| Add DiffEncode to LoCC-SNN (ablation) | LoCC-SNN · VSP tasks | operations: up to 46% | Helped |
| Use ST-BIF neurons instead of soft-reset IF neurons (ablation) | SNN · VSP tasks | task performance | Helped |
| Use DiffEncode with IF neurons instead of VaniEncode (ablation) | IF-SNN · DAVIS2017 | segmentation performance | Hurt |
| Use DiffEncode with LoCC-SNN rather than VaniEncode (ablation) | LoCC-SNN · VSP tasks | task performance | No clear effect |
| Use LoCC-SNN with DiffEncode (ablation) | VSP inference · VSP tasks | inference latency | No clear effect |
Surprising findings
- IF-SNN with DiffEncode produces erroneous segmentation at equilibrium, showing DiffEncode only works with LoCC-certified neurons.
- Var-IV shows no latency advantage on NFS240 object detection (mAP50), so the apparent early-timestep gain on DAVIS J-Mean is metric-specific.
Limitations
- Energy is estimated with a simplified model that ignores memory access.
- No latency advantage in time-steps for object detection; sustained rear-layer spiking prevents deterministic latency reduction.
- Only convolution-based backbones (YOLOv3, UniTrack) evaluated; no accuracy improvement over the ANN is claimed.
Experimental setup
Datasets: NFS OTB2015 MOT16 DAVIS2017 MOTS20
Benchmarks: NFS OTB2015 MOT16 DAVIS2017 MOTS20
Metrics: mAP50 AUC HOTA J-Means energy (pJ, SOP/MAC model) time-steps
Baselines: ANN + VaniEncode (Q-ReLU) IF-SNN + VaniEncode LoCC-SNN + VaniEncode
Builds on: SpikeZIP-TF ST-BIF neuron UniTrack YOLOv3
Closest papers by meaning
- Leveraging Asynchronous Spiking Neural Networks for Ultra Efficient Event-Based Visual Processing (AAAI 2025 · similarity 0.87)
- Finding Visual Saliency in Continuous Spike Stream (AAAI 2024 · similarity 0.86)
- Unveiling the Spatial-temporal Effective Receptive Fields of Spiking Neural Networks (NeurIPS 2025 · similarity 0.86)
- Inference-Scale Complexity in ANN-SNN Conversion for High-Performance and Low-Power Applications (CVPR 2025 · similarity 0.85)
- Brain-Inspired Spiking Neural Networks for Energy-Efficient Object Detection (CVPR 2025 · similarity 0.85)
- Continuous Spatiotemporal Events Decoupling through Spike-based Bayesian Computation (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.