Research map / Federated learning and privacy / Differential privacy
VoxPrivacy: A Benchmark for Evaluating Interactional Privacy of Speech Language Models
Yuxiang Wang, HongYu Liu, Dekun Chen, Xueyao Zhang, Zhizheng Wu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add general-task data to privacy fine-tuning (mixed-task vs privacy-only) (ablation) | Kimi-Audio-sft · MMAU avg | accuracy: 61.07 -> 62.63 | Improved |
| add general-task data to privacy fine-tuning (ablation) | Kimi-Audio-sft · Meld SER | accuracy: 50.36 -> 59.96 | Improved |
| add general-task data to privacy fine-tuning (ablation) | Kimi-Audio-sft · Librispeech test-clean | WER: 6.02 -> 1.23 | Improved |
| privacy fine-tuning Kimi-Audio on 4000h data | Kimi-Audio · VoxPrivacy Tier 2 (EN) | accuracy: 49.61 -> 83.93 | Improved |
| privacy fine-tuning Kimi-Audio | Kimi-Audio · VoxPrivacy Tier 3 (EN) | accuracy: 50.13 -> 77.57 | Improved |
| privacy fine-tuning Kimi-Audio | Kimi-Audio · VoxPrivacy Tier 1 (EN) | accuracy: 73.04 -> 88.11 | Improved |
| privacy fine-tuning Kimi-Audio | Kimi-Audio · VoxPrivacy Tier 1 (EN) | IRR: 2.40 -> 5.06 | Hurt |
| spoofing attack on Tier 2 | Kimi-Audio-sft · VoxPrivacy Tier 2 (EN) | accuracy: 83.93 -> 77.52 | Hurt |
| spoofing attack on Tier 2 | Gemini-2.0-flash · VoxPrivacy Tier 2 (EN) | accuracy: 66.10 -> 60.92 | Hurt |
| jailbreaking prompt on Tier 2 | Kimi-Audio-sft · VoxPrivacy Tier 2 (EN) | accuracy: 83.93 -> 79.79 | Hurt |
| needle-in-the-haystack context on Tier 2 | Gemini-2.0-flash · VoxPrivacy Tier 2 (ZH) | accuracy: 67.34 -> 67.45 | No clear effect |
| move from Tier 2 to Tier 3 inference | Gemini-2.5-pro · VoxPrivacy (EN) | accuracy: 76.05 -> 66.28 | Hurt |
Closest papers by meaning
- Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory (ICLR 2024 · similarity 0.89)
- Measuring Physical-World Privacy Awareness of Large Language Models: An Evaluation Benchmark (ICLR 2026 · similarity 0.88)
- MultiPriv: Benchmarking Individual-Level Privacy Reasoning in Vision-Language Models (ICML 2026 · similarity 0.87)
- Beyond Memorization: Violating Privacy via Inference with Large Language Models (ICLR 2024 · similarity 0.87)
- PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action (NeurIPS 2024 · similarity 0.87)
- Position: Contextual Integrity is Inadequately Applied to Language Models (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
