Research map / Image generation / Autoregressive and latent generation
StyleStream 2.0: Fast and Controllable Streaming Voice Style Conversion
Louis Liu, Nicholas Lee, Gopala Anumanchipalli
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Presents StyleStream 2.0, a streaming zero-shot voice style conversion system that aims to cut latency and add a text-based interface for style control, building on StyleStream 1.0.
Problem
Voice style conversion (transforming timbre, accent, and emotion of a source utterance to a target voice while preserving linguistic content) needs to be streamable with low latency and controllable via text; prior StyleStream 1.0 had about 1s end-to-end latency on consumer hardware and no text-base
Approach
Extends StyleStream 1.0 toward lower-latency streaming operation and a text-based interface for designing or controlling voice style; the abstract does not specify the mechanism.
Key results
- StyleStream 1.0 is reported to have approximately 1s end-to-end latency on consumer hardware, the baseline StyleStream 2.0 aims to improve on.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add autoregressive (Self Forcing style) fine-tuning (ablation) | StyleStream 2.0 streaming Stylizer · Voice style conversion (streaming) | WER (%): 19.54 -> 12.03 | Improved |
| Add autoregressive fine-tuning (ablation) | StyleStream 2.0 streaming Stylizer · Voice style conversion (streaming) | S-SIM / E-SIM: S-SIM 0.866 -> 0.866; E-SIM 0.767 -> 0.765 | No clear effect |
| Use perceiver-resampler style encoder instead of StyleStream 1.0 TDNN encoder (ablation) | StyleStream 2.0 offline · Voice style conversion (offline) | A-SIM: 0.633 -> 0.705 | Improved |
| Use perceiver-resampler style encoder instead of TDNN encoder (ablation) | StyleStream 2.0 offline · Voice style conversion (offline) | WER (%): 8.04 -> 7.00 | Improved |
| Replace multi-step diffusion Stylizer with few-step pixel MeanFlow Stylizer (4 NFE vs 16) | StyleStream 1.0 offline · Voice style conversion (offline) | S-SIM: 0.869 -> 0.879 | Improved |
| Replace multi-step diffusion Stylizer with pMF plus AR fine-tuning | StyleStream 1.0 streaming · Streaming, 300 ms chunk, RTX 4060 | per-chunk processing time (ms): 559.3 -> 140.7 | Improved |
| Replace multi-step diffusion Stylizer with pMF plus AR fine-tuning | StyleStream 1.0 streaming · Voice style conversion (streaming) | WER (%): 11.41 -> 12.02 | No clear effect |
| Increase streaming chunk size from 160 ms to 600 ms (ablation) | StyleStream 2.0 · Streaming, RTX 4060 | WER (%): 14.02 -> 10.87 | Improved |
| Use offline instead of streaming inference for voice design (ablation) | StyleStream 2.0 · Voice design | WER (%): 10.42 -> 6.89 | Improved |
| Use StyleStream 2.0 voice editing versus cascaded Vevo | Voice editing · Voice editing | A-pres.: 0.6303 -> 0.5102 | Hurt |
| Use StyleStream 2.0 versus Qwen3-TTS cascade for voice design | Voice design · Voice design | E-acc (%): 36.33 -> 32.20 | Hurt |
Limitations
- Full text unavailable; limitations cannot be determined from the abstract.
Experimental setup
Baselines: StyleStream 1.0
Builds on: StyleStream 1.0
Closest papers by meaning
- StableVC: Style Controllable Zero-Shot Voice Conversion with Conditional Flow Matching (AAAI 2025 · similarity 0.89)
- Vevo: Controllable Zero-Shot Voice Imitation with Self-Supervised Disentanglement (ICLR 2025 · similarity 0.89)
- Style2Talker: High-Resolution Talking Head Generation with Emotion Style and Art Style (AAAI 2024 · similarity 0.87)
- StyleSinger: Style Transfer for Out-of-Domain Singing Voice Synthesis (AAAI 2024 · similarity 0.86)
- MM-TTS: Multi-Modal Prompt Based Style Transfer for Expressive Text-to-Speech Synthesis (AAAI 2024 · similarity 0.86)
- FaceSpeak: Expressive and High-Quality Speech Synthesis from Human Portraits of Different Styles (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
