Research map / Architectures and efficiency / RNNs and normalization
Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add augmented loss to VD-LSTM (ablation) | VD-LSTM (small, 200 units) · PTB test | perplexity: 87.3 -> 82.9 | Helped |
| Reuse input embeddings in output projection (ablation) | VD-LSTM (small, 200 units) · PTB test | perplexity: 87.3 -> 85.1 | Helped |
| Add augmented loss to VD-LSTM (ablation) | VD-LSTM (medium, 650 units) · PTB test | perplexity: 77.7 -> 74.7 | Helped |
| Reuse input embeddings in output projection (ablation) | VD-LSTM (medium, 650 units) · PTB test | perplexity: 77.7 -> 73.9 | Helped |
| Add augmented loss to VD-LSTM (ablation) | VD-LSTM (large, 1500 units) · PTB test | perplexity: 72.6 -> 71.2 | Helped |
| Reuse input embeddings in output projection (ablation) | VD-LSTM (large, 1500 units) · PTB test | perplexity: 72.6 -> 69.0 | Helped |
| Add augmented loss to VD-LSTM (ablation) | VD-LSTM (small, 200 units) · Wikitext-2 test | perplexity: 105.9 -> 103.8 | Helped |
| Reuse input embeddings in output projection (ablation) | VD-LSTM (small, 200 units) · Wikitext-2 test | perplexity: 105.9 -> 100.5 | Helped |
| Add augmented loss to VD-LSTM (ablation) | VD-LSTM (medium, 650 units) · Wikitext-2 test | perplexity: 95.3 -> 93.1 | Helped |
| Reuse input embeddings in output projection (ablation) | VD-LSTM (medium, 650 units) · Wikitext-2 test | perplexity: 95.3 -> 87.7 | Helped |
| Combine augmented loss and reused embeddings (ablation) | VD-LSTM (large, 1500 units) · PTB test | perplexity: 72.6 -> 68.5 | Helped |
| Add reused embeddings to VD-RHN (ablation) | VD-RHN · PTB test | perplexity: VD-RHN by a perplexity of 2.5; 68.5 -> 66.0 | Helped |
Closest papers by meaning
- LightRNN: Memory and Computation-Efficient Recurrent Neural Networks (NeurIPS 2016 · similarity 0.90)
- A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks (ICLR 2021 · similarity 0.87)
- Memory Architectures in Recurrent Neural Network Language Models (ICLR 2018 · similarity 0.87)
- Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks (ICLR 2017 · similarity 0.87)
- Closed-Form Training Dynamics Reveal Learned Features and Linear Structure in Word2Vec-like Models (NeurIPS 2025 · similarity 0.86)
- Leviathan: Decoupling Input and Output Representations in Language Models (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.