Research map / Large language models / Language, safety and interpretability
UniSS: Unified Expressive Speech-to-Speech Translation with Your Voice
Sitong Cheng, Bianweizhen, Xinsheng Wang, Ruibin Yuan, Jianyi Chen, Shunshun Yin, Yike Guo, Wei Xue
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Phase 3 refinement training (ablation) | UniSS base (Phase 1+2) · CVSS-T EN-ZH | Speech-BLEU: 29.38 -> 30.28 (+0.90) | Improved |
| add Phase 3 refinement training (ablation) | UniSS base (Phase 1+2) · CVSS-T ZH-EN | Speech-BLEU: 21.55 -> 23.61 (+2.06) | Improved |
| add Phase 1 speech-text alignment (ablation) | UniSS · CVSS-T EN-ZH | Speech-BLEU: 22.20 -> 29.38 (-7.18 without) | Improved |
| add Phase 1 speech-text alignment (ablation) | UniSS · CVSS-T ZH-EN | Speech-BLEU: 11.40 -> 21.55 (-10.15 without) | Improved |
| use GLM-4 linguistic tokenizer instead of BiCodec semantic tokens (ablation) | UniSS · CVSS-T EN-ZH | Speech-BLEU: -15.01 | Improved |
| use GLM-4 linguistic tokenizer instead of BiCodec semantic tokens (ablation) | UniSS · CVSS-T ZH-EN | Speech-BLEU: -8.73 | Improved |
| add cross-modal chain-of-thought (intermediate text) instead of direct S2ST (ablation) | UniSS · CVSS-T EN-ZH | Speech-BLEU: -14.94 | Improved |
| add cross-modal chain-of-thought (intermediate text) instead of direct S2ST (ablation) | UniSS · CVSS-T ZH-EN | Speech-BLEU: -14.40 | Improved |
| switch from Quality mode to Performance mode | UniSS 1.5B · CVSS-T | Speed vs Speech-BLEU: 1.07x speedup with 1.84 point reduction in Speech-BLEU | Mixed |
| shrink backbone to Qwen2.5-0.5B (UniSS-Small, Performance mode) | UniSS · CVSS-T | Speedup: 1.25x speedup, Speech-BLEU 25.68 | Mixed |
| use UniSS (Q) instead of Seamless-Ex | expressive S2ST · CVSS-T | SLC 0.2 / duration consistency: 44% and 67% improvement | Improved |
| use UniSS (Q) instead of Seamless-Ex | expressive S2ST · Expressive emotion dataset | Emotion similarity MOS: 3.56 -> 4.51 | Improved |
Closest papers by meaning
- NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers (ICLR 2024 · similarity 0.87)
- High-Fidelity Simultaneous Speech-To-Speech Translation (ICML 2025 · similarity 0.86)
- Unified Synthesis of Compositional Speech and Sound from Free-Form Text Prompts (NeurIPS 2026 · similarity 0.86)
- UniCATS: A Unified Context-Aware Text-to-Speech Framework with Contextual VQ-Diffusion and Vocoding (AAAI 2024 · similarity 0.86)
- TransVIP: Speech to Speech Translation System with Voice and Isochrony Preservation (NeurIPS 2024 · similarity 0.86)
- Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
