Research map / Detection and segmentation / Open-set and few-shot detection
WhisperDiari: A Whisper-Based Speaker Diarization Framework in Token Space Leveraging Semantic and Speaker Information for Better Text Adaptability
Yongkang Yin, Yuexian Zou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| token-level joint diarization and ASR (WhisperDiari) vs separate diarization pipelines | LibriDiari n-speaker · LibriDiari n-speaker | tDER: 12.4 (best baseline 3D-Speaker) -> 8.9 | Improved |
| use WhisperDiari instead of 3D-Speaker | LibriDiari n-speaker · LibriDiari n-speaker | DER: 10.2 -> 11.9 | Hurt |
| fix speaker count in training and evaluation (ablation) | WhisperDiari · LibriDiari 2-speaker | tDER: 9.9 -> 7.3 | Improved |
| joint token-level diarization vs diarize-then-ASR pipeline | WhisperDiari · AMI (sequential pipeline) | WER: 18.1 -> 16.9 | Improved |
| use WhisperDiari instead of DiaZen | AMI (time-aligned) · AMI (time-aligned) | DER: 18.4 -> 22.7 | Hurt |
| add speaker adapter (ablation) | WhisperDiari · LibriDiari n-speaker | tDER: 15.4 -> 8.9 | Improved |
| add speaker similarity matrix supervision (Lsim) (ablation) | WhisperDiari · LibriDiari n-speaker | tDER: 10.5 -> 8.9 | Improved |
| increase Lsim comparison steps from default to 5 (ablation) | WhisperDiari · LibriDiari n-speaker | tDER: 8.9 -> 10.2 | Hurt |
Closest papers by meaning
- Sortformer: A Novel Approach for Permutation-Resolved Speaker Supervision in Speech-to-Text Systems (ICML 2025 · similarity 0.87)
- Enhancing Zero-Shot Multi-Speaker TTS with Negated Speaker Representations (AAAI 2024 · similarity 0.85)
- ALMTokenizer: A Low-bitrate and Semantic-rich Audio Codec Tokenizer for Audio Language Modeling (ICML 2025 · similarity 0.85)
- A Unified Audio Language Model with Text-Aligned Factorized Audio Tokenization (NeurIPS 2026 · similarity 0.84)
- IndicVoices-R: Unlocking a Massive Multilingual Multi-speaker Speech Corpus for Scaling Indian TTS (NeurIPS 2024 · similarity 0.84)
- TaDiCodec: Text-aware Diffusion Speech Tokenizer for Speech Language Modeling (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
