Research map / Architectures and efficiency / RNNs and normalization
Training and Evaluating Improved Dependency-Based Word Embeddings
Chen Li, Jianxin Li, Yangqiu Song, Ziwei Lin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multi-order dependency context with learned dependency weights and vectors to Skip-Gram NS | SG&NS · SIMLEX-999 | Spearman: 0.3559 -> 0.3686 | Improved |
| add multi-order dependency context to Skip-Gram NS | SG&NS · CoNLL-2003 NER | F1: 82.44 -> 85.14 | Improved |
| add multi-order dependency context to CBOW HS | CBOW&HS · CoNLL-2003 NER | F1: 80.67 -> 82.87 | Improved |
| add dependency vectors to input in addition to dependency weights (SG) (ablation) | SGDeps&NS · CoNLL-2003 NER | F1: 83.89 -> 85.14 | Improved |
| add dependency vectors to input in addition to weights (CBOW) (ablation) | CBOWDeps&HS · WS-353-SIM | Spearman: 0.7094 -> 0.7234 | Improved |
| add dependency vectors to input in addition to weights (SG) (ablation) | SGDeps&NS · RG-65 | Spearman: 0.7267 -> 0.7348 | Improved |
| multi-order SG Deps vs first-order Deps (Levy & Goldberg) | SGDeps&NS · RW | Spearman: 0.3994 -> 0.3964 | No clear effect |
| replace Deps/EXT dependency embeddings with proposed on coreference | SGDeps&NS · CoNLL-2012 coreference | MUC F1: 70.8 -> 71.3 | Improved |
| add dependency context to CBOW HS | CBOW&HS · CoNLL-2012 coreference | MUC F1: 69.2 -> 69.8 | Mixed |
Closest papers by meaning
- Learning Semantic Representations for Novel Words: Leveraging Both Form and Context (AAAI 2019 · similarity 0.89)
- Context Selection for Embedding Models (NeurIPS 2017 · similarity 0.89)
- A Simple but Tough-to-Beat Baseline for Sentence Embeddings (ICLR 2017 · similarity 0.88)
- Learning Context-Specific Word/Character Embeddings (AAAI 2017 · similarity 0.88)
- By Tying Embeddings You Are Assuming the Distributional Hypothesis (ICML 2024 · similarity 0.87)
- Repurposing Language Models into Embedding Models: Finding the Compute-Optimal Recipe (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
