Research map / Architectures and efficiency / RNNs and normalization
Learning Semantic Representations for Novel Words: Leveraging Both Form and Context
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| combine surface-form and context embeddings rather than use only surface-form information (ablation) | Form-context model · Definitional Nonce test | MRR: 0.12982 -> 0.17537 | Helped |
| combine surface-form and context embeddings rather than use only context information (ablation) | Form-context model · Definitional Nonce test | MRR: 0.06560 -> 0.17537 | Helped |
| use an embedding-conditioned gate instead of a single shared composition coefficient (ablation) | Form-context model · Definitional Nonce test | MRR: 0.16200 -> 0.17537 | Helped |
| use an embedding-conditioned gate instead of a single shared composition coefficient (ablation) | Form-context model · Definitional Nonce test | Median Rank: 55 -> 49 | Helped |
| use a multi-layer neural network or component-wise weighting for composition (ablation) | Form-context model · Reported experiments | performance | No clear effect |
| refine the composition iteratively over multiple iterations (ablation) | Form-context model · Reported experiments | performance | No clear effect |
| combine context and form information instead of using surface-form information alone (ablation) | Form-context model · Contextual Rare Words | Spearman's rho | Mixed |
| use the gated form-context model rather than surface-form only (ablation) | Form-context model trained on fastText embeddings · Contextual Rare Words, 128 contexts | Spearman's rho: 0.501 -> 0.508 | Helped |
| use form-context OOV embeddings instead of skipgram without OOV words | Bag-of-words sentence classifier · SentEval sentence-level classification tasks (average) | accuracy: 74.98 -> 75.34 | Helped |
| increase available contexts to 16 or more | Form-context model · Contextual Rare Words | Spearman's rho | Helped |
| increase available contexts to 32 or more | Form-context model · Contextual Rare Words | performance relative to surface-form embeddings | No clear effect |
Closest papers by meaning
- Learning Context-Specific Word/Character Embeddings (AAAI 2017 · similarity 0.92)
- Improving Word Embeddings with Convolutional Feature Learning and Subword Information (AAAI 2017 · similarity 0.90)
- A Simple but Tough-to-Beat Baseline for Sentence Embeddings (ICLR 2017 · similarity 0.90)
- Understanding the Semantic Content of Sparse Word Embeddings Using a Commonsense Knowledge Base (AAAI 2020 · similarity 0.90)
- All Word Embeddings from One Embedding (NeurIPS 2020 · similarity 0.90)
- Definition Modeling: Learning to Define Word Embeddings in Natural Language (AAAI 2017 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.