Research map / Vision-language and multimodal / Cross-modal learning
Unsupervised Cross-Modal Alignment of Speech and Text Embedding Spaces
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace parallel-dictionary alignment with unsupervised alignment | Speech2Vec + Word2Vec · Spoken word classification, EN_ls-en_ls | accuracy: 25.4 -> 23.7 | Hurt |
| Replace parallel-dictionary alignment with unsupervised alignment | Speech2Vec + Word2Vec · Spoken word synonyms retrieval, EN_ls-en_ls | P@1: 52.6 -> 43.2 | Hurt |
| Replace parallel-dictionary alignment with unsupervised alignment | Speech2Vec + Word2Vec · Spoken word translation, EN_ls-fr_ls | P@1: 47.9 -> 40.5 | Hurt |
| Use unsupervised k-means grouping instead of word identities (ablation) | Configuration A · Spoken word classification, EN_ls-en_ls | accuracy: 23.7 -> 19.4 | Hurt |
| Use BES-GMM segmentation and k-means grouping instead of forced alignment and word identities (ablation) | Configuration A · Spoken word classification, EN_ls-en_ls | accuracy: 23.7 -> 10.9 | Hurt |
| Use ES-KMeans segmentation and k-means grouping instead of forced alignment and word identities (ablation) | Configuration A · Spoken word classification, EN_ls-en_ls | accuracy: 23.7 -> 11.5 | Hurt |
| Use SylSeg segmentation and k-means grouping instead of forced alignment and word identities (ablation) | Configuration A · Spoken word classification, EN_ls-en_ls | accuracy: 23.7 -> 6.5 | Hurt |
| Use equally sized chunks and k-means grouping instead of forced alignment and word identities (ablation) | Configuration A · Spoken word classification, EN_ls-en_ls | accuracy: 23.7 -> 0.8 | Hurt |
| Replace linear alignment mapping with a multilayer neural network (ablation) | Unsupervised cross-modal alignment · Evaluation tasks | performance | No clear effect |
Closest papers by meaning
- Comparing Unsupervised Word Translation Methods Step by Step (NeurIPS 2019 · similarity 0.90)
- Normalization of Language Embeddings for Cross-Lingual Alignment (ICLR 2022 · similarity 0.89)
- Word translation without parallel data (ICLR 2018 · similarity 0.89)
- Unsupervised Hyper-alignment for Multilingual Word Embeddings (ICLR 2019 · similarity 0.89)
- Geometric Alignment without Functional Equivalence: A Layer-wise Analysis of the Speech-Text Modality Gap (NeurIPS 2026 · similarity 0.87)
- Strong Helps Weak: Directional Cross-Modal Alignment Transfer in Multi-modal LLMs (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.