Research map / Architectures and efficiency / RNNs and normalization
Parallel Training of GRU Networks with a Multi-Grid Solver for Long Sequences
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use parallel-in-time MGRIT training instead of serial training | Implicit GRU · HMDB51 full dataset | training speedup: $6.5\times$ speedup | Helped |
| Use parallel-in-time MGRIT training instead of serial training | Implicit GRU · UCI-HAR | training speedup: roughly $2\times$ speedup | Helped |
| Use Implicit GRU instead of Classic GRU | GRU · UCI-HAR | test accuracy | No clear effect |
| Use MGRIT-trained Implicit GRU with parallel inference rather than serial inference | Implicit GRU · HMDB51 subset | test accuracy | No clear effect |
| Set MGRIT coarsening factor to 2 rather than 4 or 8 (ablation) | MGRIT training · HMDB51 subset | residual convergence | Helped |
| Set MGRIT coarsening factor to 8 rather than 4 (ablation) | MGRIT training · HMDB51 subset | training robustness | Hurt |
| Use more than 2 forward and 1 backward MGRIT iterations (ablation) | MGRIT training · HMDB51 subset | training performance | No clear effect |
| Increase MGRIT parallelism to 64 MPI ranks (ablation) | Implicit GRU · HMDB51 full dataset | training runtime | No clear effect |
| Increase MPI ranks beyond 32 (ablation) | MGRIT training · UCI-HAR | training speedup | No clear effect |
Closest papers by meaning
- Parallelizing non-linear sequential models over the sequence length (ICLR 2024 · similarity 0.88)
- Sequence Accumulation and Beyond: Infinite Context Length on Single GPU and Large Clusters (AAAI 2025 · similarity 0.85)
- FlashRNN: I/O-Aware Optimization of Traditional RNNs on modern hardware (ICLR 2025 · similarity 0.85)
- Parallel-in-Time Training of Recurrent Neural Networks for Dynamical Systems Reconstruction (NeurIPS 2026 · similarity 0.85)
- Harnessing Manycore Processors with Distributed Memory for Accelerated Training of Sparse and Recurrent Models (AAAI 2024 · similarity 0.85)
- GSPN-2: Efficient Parallel Sequence Modeling (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.