Research map / Large language models / Language, safety and interpretability
Neural Language Modeling by Jointly Learning Syntax and Lexicon
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PRPN instead of LayerNorm HyperNetworks | Penn Treebank character-level language modeling · Penn Treebank test | BPC: 1.23 -> 1.202 | Helped |
| use PRPN instead of the strongest listed baseline | Penn Treebank word-level language modeling · Penn Treebank test | PPL: 58.3 -> 61.98 | Hurt |
| use PRPN instead of LSTM + continuous cache pointer | Text8 language modeling · Text8 valid | PPL: 99.9 -> 81.64 | Helped |
| use PRPN instead of UML-DOP | unsupervised constituency parsing · WSJ10 | UF1: 82.9 -> 70.02 | Hurt |
| add the Parsing Network and structured attention rather than using normal attention (ablation) | PRPN · Penn Treebank test | PPL: 64.42 -> 61.98 | Helped |
| add Structured Attention to the Reading Network (ablation) | PRPN · Penn Treebank test | PPL: 64.63 -> 61.98 | Helped |
| add Structured Attention to the Predict Network (ablation) | PRPN · Penn Treebank test | PPL: 63.65 -> 61.98 | Helped |
| add the Parsing Network and Structured Attention to both Reading and Predict Networks (ablation) | 2-layer LSTM baseline · Penn Treebank test | PPL: 65.81 -> 61.98 | Helped |
Closest papers by meaning
- Unraveling Syntax: Language Modeling and the Substructure of Grammars (ICML 2026 · similarity 0.88)
- Towards a theory of how the structure of language is acquired by deep neural networks (NeurIPS 2024 · similarity 0.86)
- Learning Language Structures Through Grounding (AAAI 2025 · similarity 0.85)
- Training Neural Networks as Recognizers of Formal Languages (ICLR 2025 · similarity 0.85)
- Dynamic Syntactic Feature Filtering and Injecting Networks for Cross-lingual Dependency Parsing (AAAI 2025 · similarity 0.85)
- Improving Unsupervised Constituency Parsing via Maximizing Semantic Information (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.