Research map / Architectures and efficiency / RNNs and normalization
Learned in Translation: Contextualized Word Vectors
Bryan McCann, James Bradbury, Caiming Xiong, Richard Socher
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add CoVe-L to GloVe inputs (ablation) | BCN classification models · SST-2, SST-5, IMDb, TREC-6, TREC-50, SNLI, and SQuAD validation | accuracy (classification); F1 (SQuAD): SST-2: 88.4 -> 91.1; SST-5: 53.5 -> 54.5; IMDb: 91.1 -> 91.7; TREC-6: 94.9 -> 95.8; TREC-50: 89.2 -> 90.5; SNLI: 87.7 -> | Improved |
| Add character n-gram embeddings alongside GloVe and CoVe-L (ablation) | BCN classification models and DCN question answering model · SST-2, SST-5, IMDb, TREC-6, TREC-50, SNLI, and SQuAD validation | accuracy (classification); F1 (SQuAD): SST-2: 91.1 -> 91.2; SST-5: 54.5 -> 55.2; IMDb: 91.7 -> 92.1; TREC-6: 95.8 -> 95.8; TREC-50: 90.5 -> 91.2; SNLI: 87.9 -> | Improved |
| Use CoVe-L rather than CoVe-S (ablation) | GloVe-based downstream models · SST-2, SST-5, IMDb, TREC-6, TREC-50, SNLI, and SQuAD validation | accuracy (classification); F1 (SQuAD): SST-2: 89.0 -> 91.1; SST-5: 54.0 -> 54.5; IMDb: 90.6 -> 91.7; TREC-6: 94.7 -> 95.8; TREC-50: 89.6 -> 90.5; SNLI: 87.3 -> | Mixed |
| Replace CoVe-L with character n-gram embeddings (ablation) | GloVe-based downstream models · SST-2 validation | accuracy: 91.1 -> 90.1 | Hurt |
| Replace CoVe-L with character n-gram embeddings (ablation) | GloVe-based downstream models · SST-5 validation | accuracy: 54.5 -> 52.2 | Hurt |
| Replace CoVe-L with character n-gram embeddings (ablation) | GloVe-based downstream models · IMDb validation | accuracy: 91.7 -> 91.3 | Hurt |
| Replace CoVe-L with character n-gram embeddings (ablation) | GloVe-based downstream models · TREC-6 validation | accuracy: 95.8 -> 94.7 | No clear effect |
| Replace CoVe-L with character n-gram embeddings (ablation) | GloVe-based downstream models · TREC-50 validation | accuracy: 90.5 -> 89.8 | Hurt |
| Replace CoVe-L with character n-gram embeddings (ablation) | GloVe-based downstream models · SNLI validation | accuracy: 87.9 -> 87.7 | No clear effect |
| Replace CoVe-L with character n-gram embeddings (ablation) | GloVe-based DCN · SQuAD validation | F1: 79.5 -> 78.1 | Hurt |
| Use CoVe-L instead of skip-thought vectors with GloVe and character embeddings (ablation) | classification models · SST-2, SST-5, TREC-6, TREC-50, and SNLI validation | accuracy: SST-2: 88.7 -> 91.2; SST-5: 52.1 -> 55.2; TREC-6: 94.2 -> 95.8; TREC-50: 89.6 -> 91.2; SNLI: 86.0 -> 88.1 | Improved |
Closest papers by meaning
- Deep contextualized word representations (ICLR 2018 · similarity 0.89)
- VAST: The Valence-Assessing Semantics Test for Contextualizing Language Models (AAAI 2022 · similarity 0.86)
- What do you learn from context? Probing for sentence structure in contextualized word representations (ICLR 2019 · similarity 0.86)
- Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning (ICLR 2018 · similarity 0.86)
- Improving Context-Aware Neural Machine Translation Using Self-Attentive Sentence Embedding (AAAI 2020 · similarity 0.86)
- Prompt Compression with Context-Aware Sentence Encoding for Fast and Improved LLM Inference (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
