Research map / Large language models / Language, safety and interpretability
Deep contextualized word representations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add ELMo to the baseline model | SQuAD baseline · SQuAD test | F1: 81.1 -> 85.8 | Helped |
| Add ELMo to the baseline model | SNLI ESIM · SNLI test | accuracy: 88.0 -> 88.7 | Helped |
| Add ELMo to the baseline model | SRL baseline · SRL CoNLL 2012 test | F1: 81.4 -> 84.6 | Helped |
| Add ELMo to the baseline model | Coreference baseline · Coref CoNLL 2012 test | average F1: 67.2 -> 70.4 | Helped |
| Add ELMo to the baseline model | NER baseline · CoNLL 2003 NER test | F1: 90.15 -> 92.22 | Helped |
| Add ELMo to the baseline model | SST-5 baseline · SST-5 test | accuracy: 51.4 -> 54.7 | Helped |
| Use all biLM layers rather than only the top layer (ablation) | ELMo task models · SQuAD development | F1: 84.7 -> 85.0 | Helped |
| Learn individual weights over all biLM layers rather than averaging them (ablation) | ELMo task models · SQuAD development | F1: 85.0 -> 85.2 | Helped |
| Include ELMo at both input and output instead of input only (ablation) | ELMo task model · SNLI development | accuracy: 88.9 -> 89.5 | Helped |
| Include ELMo at both input and output instead of input only (ablation) | ELMo task model · SQuAD development | F1: 85.1 -> 85.6 | Helped |
| Include ELMo at both input and output rather than input only (ablation) | ELMo task model · SRL development | F1: 84.7 -> 84.3 | Hurt |
| Add fine tuning of the biLM on task data (ablation) | ELMo + ESIM · SNLI development | accuracy: 88.9% -> 89.5% | Helped |
Closest papers by meaning
- What do you learn from context? Probing for sentence structure in contextualized word representations (ICLR 2019 · similarity 0.91)
- VAST: The Valence-Assessing Semantics Test for Contextualizing Language Models (AAAI 2022 · similarity 0.90)
- Learning Semantic Representations for Novel Words: Leveraging Both Form and Context (AAAI 2019 · similarity 0.90)
- Learning Context-Specific Word/Character Embeddings (AAAI 2017 · similarity 0.89)
- Contextual Document Embeddings (ICLR 2025 · similarity 0.87)
- Closed-Form Training Dynamics Reveal Learned Features and Linear Structure in Word2Vec-like Models (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.