Research map / Architectures and efficiency / RNNs and normalization
Efficient recurrent architectures through activity sparsity and sparse back-propagation through time
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace GRU units with event-generating EGRU units (threshold-triggered sparse communication) | GRU on DVS Gesture · DVS Gesture | test accuracy: 88.0% vs 88.1% | No clear effect |
| Add data augmentation to EGRU and tune CNN head, DropConnect, Zoneout | EGRU on DVS Gesture · DVS Gesture | test accuracy: 97.8% vs 97.4% for CNN+GRU+DA | Helped |
| Use scalar threshold shared across all units (ablation) | EGRU (1024) on DVS Gesture · DVS Gesture | accuracy / activity sparsity: accuracy 88.3 vs 90.2; sparsity 90.8 vs 82.5 | Mixed |
| Remove the -y term (no reset) from the event equation (ablation) | EGRU (1024) on DVS Gesture · DVS Gesture | accuracy: 88.9 vs 90.2 | Hurt |
| Use hard reset of internal state on event (ablation) | EGRU (1024) on DVS Gesture · DVS Gesture | accuracy: 87.2 vs 90.2 | Hurt |
| Remove regularization losses on DVS Gesture (ablation) | EGRU (1024) on DVS Gesture · DVS Gesture | accuracy / activity sparsity: 89.3 vs 90.2; sparsity 76.5 vs 82.5 | Hurt |
| Event-based GRU (EGRU) with threshold activity sparsity | GRU on sequential MNIST · sMNIST | test accuracy: 98.3% vs 98.8% | Hurt |
| Event-based GRU (EGRU) with threshold activity sparsity | GRU on permuted sequential MNIST · psMNIST | test accuracy: 95.1% (EGRU) reported; GRU baseline not in table | No clear effect |
| Event-based GRU (EGRU) with 1350 units | GRU on PennTreebank · PTB | test perplexity: 57.2 vs 66.3 (lower is better) | Helped |
| Event-based GRU (EGRU) with 1350 units | GRU on WikiText-2 · WikiText-2 | test perplexity: 70.6 vs 71.8 (lower is better) | Helped |
| Increase EGRU hidden size from 1350 to 2000 units | EGRU on WikiText-2 · WikiText-2 | test perplexity: 68.9 vs 70.6 (lower is better) | Helped |
Closest papers by meaning
- Exploiting Symmetric Temporally Sparse BPTT for Efficient RNN Training (AAAI 2024 · similarity 0.90)
- Sparse Spiking Neural Network: Exploiting Heterogeneity in Timescales for Pruning Recurrent SNN (ICLR 2024 · similarity 0.89)
- Accelerating Linear Recurrent Neural Networks for the Edge with Unstructured Sparsity (ICML 2025 · similarity 0.89)
- Recurrent neural networks: vanishing and exploding gradients are not the end of the story (NeurIPS 2024 · similarity 0.89)
- Emergent mechanisms for long timescales depend on training curriculum and affect performance in memory tasks (ICLR 2024 · similarity 0.89)
- Recurrent neural network dynamical systems for biological vision (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.