Research map / Label-efficient and robust learning / Recognition and long-tail
Style-transfer and Paraphrase: Looking for a Sensible Semantic Similarity Metric
Ivan P. Yamshchikov, Viacheslav Shibaev, Nikolay Khlebnikov, Alexey Tikhonov
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use WMD rather than the other evaluated metrics to estimate semantic similarity | Paraphrase and style-transfer evaluation · 14 datasets | correlation with human evaluation: 0.92 | Improved |
| Use Word2Vec cosine similarity as a semantic-preservation metric | Paraphrase and style-transfer evaluation · 14 datasets | correlation with human evaluation: 0.46 | Hurt |
| Use FastText cosine similarity as a semantic-preservation metric | Paraphrase and style-transfer evaluation · 14 datasets | correlation with human evaluation: 0.52 | Hurt |
| Use BLEU for quantitative semantic similarity estimates rather than binary separation of random pairs | Paraphrase and style-transfer evaluation · Random sentence pairs across datasets | variability on random sentences: 0.2% | Mixed |
| Treat sentiment transfer as ordinary style transfer while assuming sentiment is not semantic | Human assessment of Yelp! rewrites · Yelp! rewrite | human labeling: 2.68 ± 0.83 | Hurt |
Closest papers by meaning
- Semantic-KG: Using Knowledge Graphs to Construct Benchmarks for Measuring Semantic Similarity (NeurIPS 2025 · similarity 0.86)
- ParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfer (AAAI 2024 · similarity 0.86)
- A Massive Scale Semantic Similarity Dataset of Historical English (NeurIPS 2023 · similarity 0.85)
- MT-Ranker: Reference-free machine translation evaluation by inter-system ranking (ICLR 2024 · similarity 0.83)
- Sentence-level Aggregation of Lexical Metrics Correlates Stronger with Human Judgements than Corpus-level Aggregation (AAAI 2025 · similarity 0.83)
- Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
