Research map / Architectures and efficiency / RNNs and normalization
Flow Equivariant Recurrent Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces flow equivariance for sequence models and FERNNs, recurrent networks whose hidden state is lifted to a bank of flow generators so they are equivariant to time-parameterized symmetries like visual motion.
Problem
Standard RNNs and other sequence models are not equivariant to time-parameterized symmetry transformations (flows such as visual motion), limiting generalization in sequence modeling.
Approach
Define flow equivariance for sequence models and build Flow Equivariant RNNs by lifting the hidden state to a product group V x G (flow generators times the spatial group) and applying the one-step flow action psi_1(nu) to each flow channel in the recurrence, so moving inputs are processed in their own reference frame.
Key results
- FERNNs reach nearly an order of magnitude lower next-step prediction MSE than G-RNNs on Flowing MNIST and train in far fewer iterations.
- On length generalization to 70-step forward prediction, G-RNN predictions degrade after the training length while FERNN-V^T_2 remains consistent.
- FERNNs generalize zero-shot to unseen higher velocities (e.g. trained on V^T_1, tested on V^T_2) where G-RNNs fail to forward predict.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use flow-equivariant recurrence instead of a G-RNN | Next-step prediction on Flowing MNIST · Flowing MNIST translation and rotation | MSE | Helped |
| Use flow-equivariant recurrence instead of a G-RNN | Next-step prediction on Flowing MNIST · Flowing MNIST | training speed | Helped |
| Use flow-equivariant recurrence instead of a G-RNN | Next-step prediction models · Flowing MNIST translation | length generalization error | Helped |
| Use flow-equivariant recurrence instead of a G-RNN | Next-step prediction models · Flowing MNIST rotation | length generalization | Mixed |
| Train on low velocities with a lifted set containing higher velocities | FERNN · Flowing MNIST translation and rotation velocity generalization | test MSE | Helped |
| Use flow-equivariant recurrence instead of a G-RNN | Next-step prediction models · Flowing MNIST velocities outside the integrated set | test MSE | Helped |
| Use flow-equivariant recurrence instead of non-flow-equivariant models | KTH action recognition models · KTH action recognition with camera-motion flows | accuracy | Helped |
| Use flow-equivariant recurrence instead of non-flow-equivariant counterparts | KTH action recognition models · KTH action recognition, V_train = V_test = V^T_2 | accuracy | Helped |
| Add partial flow equivariance | G-RNN next-step prediction · Flowing MNIST translation, FERNN-V^T_1 evaluated on V^T_2 | MSE | Helped |
| Replace the prescribed flow action on each lifted hidden index with a learned convolutional transformation (ablation) | FERNN-V_2^T (G-RNN+ ablation) · KTH action recognition | accuracy | Hurt |
Surprising findings
- Standard G-RNNs are not flow equivariant even when all sub-components are individually group equivariant, and this extends to most existing sequence models including transformers and state space models (per the authors).
Limitations
- Restricted to constant-velocity flows; variable velocity handled only empirically via inter-V kernel mixing.
- Compute and memory scale linearly with the size of the discretized velocity set V, and truncation of V breaks exact equivariance.
- Interpolation on rotated pixel grids breaks exact equivariance, weakening length generalization on Rotating MNIST.
Experimental setup
Datasets: MNIST KTH action recognition
Benchmarks: Flowing MNIST KTH action recognition
Metrics: mean squared error classification accuracy
Baselines: G-RNN G-RNN+ 3D-CNN
Builds on: group equivariant CNN G-RNN escnn
Closest papers by meaning
- Approximately Equivariant Neural Processes (NeurIPS 2024 · similarity 0.90)
- A Tale of Two Symmetries: Exploring the Loss Landscape of Equivariant Models (NeurIPS 2025 · similarity 0.89)
- A Characterization Theorem for Equivariant Networks with Point-wise Activations (ICLR 2024 · similarity 0.89)
- Improving Equivariant Networks with Probabilistic Symmetry Breaking (ICLR 2025 · similarity 0.88)
- Translation Equivariant Transformer Neural Processes (ICML 2024 · similarity 0.88)
- Lie Group Decompositions for Equivariant Neural Networks (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.