Research map / Architectures and efficiency / RNNs and normalization
Understanding Composition of Word Embeddings via Tensor Decomposition
Abraham Frandsen, Rong Ge
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use weighted tensor composition instead of simple addition (ablation) | Adjective-noun phrase embeddings · Adjective-noun phrase similarity | Spearman and Pearson correlation: rw: .446, .438 -> .460, .465; glove: .357, .336 -> .368, .347; cbow Spearman: .471 -> .474; cbow Pearson: .452 -> .471 | Improved |
| Add tensor correction to SIF phrase embeddings (ablation) | Adjective-noun phrase embeddings · Adjective-noun phrase similarity | Spearman and Pearson correlation: rw: .482, .477 -> .482, .481; glove: .429, .434 -> .433, .437; cbow: .489, .482 -> .492, .484 | Improved |
| Use unweighted tensor composition rather than simple addition (ablation) | Phrase embeddings · Phrase similarity before tuning tensor weight | Correlation with human ratings | Hurt |
| Use weighted tensor correction with tuned alpha (ablation) | Verb-object phrase embeddings · Verb-object phrase similarity | Spearman and Pearson correlation: GloVe Spearman: .401 -> .410; GloVe Pearson: .404 -> .420; CBOW Pearson: .415 -> .422; RAND-WALK weighted2: .392, .387 v | Mixed |
| Add tensor correction to SIF phrase embeddings (ablation) | Verb-object phrase embeddings · Verb-object phrase similarity | Spearman and Pearson correlation: GloVe: .387, .380 -> .411, .409; RAND-WALK: .378, .351 -> .378, .363; CBOW: .404, .404 -> .420, .417 | Mixed |
| Add adjective-noun tensor corrections to review embeddings (ablation) | Movie-review sentiment classifier · Pang and Lee movie reviews | Test accuracy: 0.741 (0.018) -> 0.759 (0.025) | No clear effect |
| Add adjective-noun tensor corrections to review embeddings (ablation) | Movie-review sentiment classifier · Large Movie Review | Test accuracy: 0.793 -> 0.794 | No clear effect |
| Use tensor rather than additive phrase composition (ablation) | Qualitative retrieval of phrase-related words · Adjective-noun and verb-object phrase retrieval | Qualitative relevance | Mixed |
Closest papers by meaning
- Further results on latent discourse models and word embeddings (JMLR 2021 · similarity 0.88)
- A Simple but Tough-to-Beat Baseline for Sentence Embeddings (ICLR 2017 · similarity 0.88)
- On the Dimensionality of Word Embedding (NeurIPS 2018 · similarity 0.88)
- CoVeR: Learning Covariate-Specific Vector Representations with Tensor Decompositions (ICML 2018 · similarity 0.87)
- TNCSE: Tensor Norm Constraints for Unsupervised Contrastive Learning of Sentence Embeddings (AAAI 2025 · similarity 0.86)
- By Tying Embeddings You Are Assuming the Distributional Hypothesis (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
