Research map / Architectures and efficiency / RNNs and normalization
Regularizing Towards Permutation Invariance In Recurrent Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SIRE regularization to an RNN | ModelNet40 point-cloud classification · ModelNet40, 100 pts | accuracy: 0.825 -> 0.835 | Helped |
| Add SIRE regularization to an RNN | ModelNet40 point-cloud classification · ModelNet40, 1000 pts | accuracy: 0.872 -> 0.878 | Helped |
| Use SIRE instead of DeepSets | ModelNet40 point-cloud classification · ModelNet40, 5000 pts | accuracy: 0.90 vs 0.899 | Mixed |
| Add SIRE regularization with λ = 0.01 | RNN · Half-range, sequence length 10 | accuracy: 0.9346 (0.006) -> 0.9584 (0.008) | Helped |
| Add SIRE regularization with λ = 0.01 | RNN · Half-range, sequence length 15 | accuracy: 0.9461 (0.001) -> 0.9658 (0.008) | Helped |
| Add SIRE regularization with λ = 0.01 | RNN · Half-range, sequence length 20 | accuracy: 0.9678 (0.005) -> 0.9780 (0.004) | Helped |
| Add SIRE regularization to the GRU | GRU · Locally Perturbed MNIST | accuracy: 0.833 -> 0.977 | Helped |
| Add SIRE regularization to an RNN | Arithmetic sum task · Sequences of integers, lengths 5 to 30 | zero-one error | Helped |
| Add SIRE regularization to an RNN | Arithmetic range task · Sequences of integers, lengths 5 to 30 | zero-one error | Mixed |
| Add SIRE regularization to an RNN | Arithmetic variance task · Sequences of integers, lengths 5 to 30 | RMSE | Helped |
| Use π-SGD for the variance task | Permutation-invariant sequence modeling · Arithmetic variance task | training error | Hurt |
| Use a recurrent model rather than DeepSets for parity | Parity classification · Boolean sequences, training lengths up to 10 and test lengths up to 100 | accuracy | Helped |
Closest papers by meaning
- Set Norm and Equivariant Skip Connections: Putting the Deep in Deep Sets (ICML 2022 · similarity 0.87)
- Permutation Equivariance of Transformers and Its Applications (CVPR 2024 · similarity 0.86)
- Learning Conditional Invariances through Non-Commutativity (ICLR 2024 · similarity 0.85)
- A Unified Analysis of Stochastic Gradient Descent with Arbitrary Data Permutations and Beyond (NeurIPS 2025 · similarity 0.85)
- Translation Equivariant Transformer Neural Processes (ICML 2024 · similarity 0.85)
- Flow Equivariant Recurrent Neural Networks (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.