Research map / Probabilistic and scientific modeling / Molecules and proteins
Hierarchical Multiscale Recurrent Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the HM-LSTM hierarchical multiscale architecture instead of a standard LSTM | IAM-OnDB sequence generation · IAM-OnDB test | Average Log-Likelihood: 1081 -> 1137 | Helped |
| Add slope annealing to HM-LSTM (ablation) | HM-LSTM · IAM-OnDB test | Average Log-Likelihood: 1137 -> 1167 | Helped |
| Use step boundary decisions instead of sampling (ablation) | LayerNorm HM-LSTM · Penn Treebank test | BPC: 1.27 -> 1.25 | Helped |
| Use step boundary decisions instead of soft boundary decisions (ablation) | LayerNorm HM-LSTM · Penn Treebank test | BPC: 1.27 -> 1.25 | Helped |
| Add slope annealing to step-function boundary decisions (ablation) | LayerNorm HM-LSTM · Penn Treebank test | BPC: 1.25 -> 1.24 | Helped |
| Add layer normalization (ablation) | HM-LSTM · Text8 test | BPC: 1.32 -> 1.29 | Helped |
| Add layer normalization (ablation) | HM-LSTM · Hutter Prize Wikipedia test | BPC: 1.34 -> 1.32 | Helped |
| Add slope annealing to HM-LSTM (ablation) | LayerNorm HM-LSTM · Penn Treebank test | BPC: 1.25 -> 1.24 | Helped |
Closest papers by meaning
- Learning to Adaptively Scale Recurrent Neural Networks (AAAI 2019 · similarity 0.87)
- Hierarchically Gated Recurrent Neural Network for Sequence Modeling (NeurIPS 2023 · similarity 0.87)
- Learning Multiscale Transformer Models for Sequence Generation (ICML 2022 · similarity 0.86)
- Learning Hierarchical Structures with Differentiable Nondeterministic Stacks (ICLR 2022 · similarity 0.86)
- State-Frequency Memory Recurrent Neural Networks (ICML 2017 · similarity 0.85)
- Quasi-Recurrent Neural Networks (ICLR 2017 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.