Research map / Probabilistic and scientific modeling / Molecules and proteins
Modeling Hierarchical Structures with Continuous Recursive Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use continuous latent hierarchical composition with parallel updates | CRvNN · ListOps | accuracy | Helped |
| Use CRvNN rather than the strongest reported baseline | CRvNN · MNLI matched test | accuracy: 70.7_3 -> 72.9_2 | Helped |
| Use CRvNN rather than Ordered Memory | CRvNN · ListOps | accuracy: 99.97\pm0.014 -> 99.6\pm0.3 | Hurt |
| Use CRvNN rather than Ordered Memory | CRvNN · ListOps, training time for 50 samples, sequence length 81-100 | training time: 3.28 min -> 0.20 min | Helped |
| Replace modulated sigmoid with simple sigmoid (ablation) | CRvNN · ListOps | accuracy: 99.6\pm0.3 -> 90.8_{163} | Hurt |
| Add modulated sigmoid instead of simple sigmoid (ablation) | CRvNN · Logical Inference, 10 operations | accuracy: 95.0_{3} -> 95.0_6 | No clear effect |
| Remove structural bias and use left-to-right recurrence (ablation) | CRvNN · ListOps | accuracy: 99.6_3 -> 83.4_{11} | Hurt |
| Add structural bias rather than use left-to-right recurrence (ablation) | CRvNN · Logical Inference, 10 operations | accuracy: 87.2_{11} -> 95.0_6 | Helped |
| Remove transition features (ablation) | CRvNN · ListOps | accuracy: 99.6_3 -> 99.4_5 | No clear effect |
| Remove halt penalty (ablation) | CRvNN · ListOps | accuracy: 99.6_3 -> 99.4_2 | No clear effect |
| Replace GeLU with ReLU (ablation) | CRvNN · ListOps | accuracy: 99.6_3 -> 99.2_4 | No clear effect |
| Replace gated recursive cell with LSTMCell (ablation) | CRvNN · ListOps | accuracy: 99.6_3 -> 71.2_{41} | Hurt |
Closest papers by meaning
- Dynamic Compositionality in Recursive Neural Networks with Structure-Aware Tag Representations (AAAI 2019 · similarity 0.89)
- Learning Hierarchical Structures with Differentiable Nondeterministic Stacks (ICLR 2022 · similarity 0.89)
- Learning to Compose Task-Specific Tree Structures (AAAI 2018 · similarity 0.87)
- RNNs implicitly implement tensor-product representations (ICLR 2019 · similarity 0.86)
- Hierarchically Gated Recurrent Neural Network for Sequence Modeling (NeurIPS 2023 · similarity 0.86)
- A Tensor Decomposition Perspective on Second-order RNNs (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.