Research map / Large language models / Language, safety and interpretability
SwitchLingua: The First Large-Scale Multilingual and Multi-Ethnic Code-Switching Dataset
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces LinguaMaster, a multi-agent LLM framework for synthesizing code-switching text, uses it to build SwitchLingua (420K CS text samples, 80+ hours of audio across 12 languages), and proposes SAER, a semantic-aware ASR error metric.
Problem
Existing code-switching datasets are small, low in language and speaker diversity, and often unnatural; standard ASR metrics like WER/CER are insensitive to semantic equivalence and transliteration variants in code-switched speech.
Approach
A generate-evaluate-refine multi-agent pipeline grounded in linguistic constraints (Poplack's free-morpheme and equivalence constraints) and tool retrieval via MCP synthesizes CS text, which is then recorded by diverse speakers; SAER combines LaBSE semantic cosine distance with language-specific WER/CER.
Key results
- SwitchLingua scores 85.4 human / 80.0 LLM overall versus 63.2/60.0 for the next-best dataset (SEAME), a 25.5% relative improvement.
- Whisper-Large-v3 achieves the best SAER in 8 of 12 evaluated languages, e.g., SAER 0.0992 on Mandarin-English versus 0.0404 for Seamless-M4T-Large-v2 (Seamless is best on Mandarin).
- The ablation shows the full LinguaMaster (LP + MAC + TI) raises the overall score from 34.6 to 62.0 human (41.7% gain over baseline GPT-4o).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Linguistic Principles (LP) (ablation) | LinguaMaster with GPT-4o baseline · LinguaMaster synthetic data quality | Overall (human/LLM): 34.6/39.6 -> 41.1/46.0 | Helped |
| add Multi-Agent Collaboration (MAC) to LP (ablation) | LinguaMaster with GPT-4o baseline · LinguaMaster synthetic data quality | Overall (human/LLM): 41.1/46.0 -> 51.6/52.0 | Helped |
| add Tool Integration (TI) to LP and MAC (ablation) | LinguaMaster with GPT-4o baseline · LinguaMaster synthetic data quality | Overall (human/LLM): 51.6/52.0 -> 62.0/65.1 | Helped |
| clean extraneous text from Qwen2-Audio-7B-Instruct outputs | Qwen2-Audio-7B-Instruct · SwitchLingua ASR evaluation | WER: 6.75% | Helped |
| clean extraneous text from Qwen2-Audio-7B-Instruct outputs | Qwen2-Audio-7B-Instruct · SwitchLingua ASR evaluation | SAER: 5.34% | Helped |
Surprising findings
- Qwen2-Audio-7B-Instruct tends to overproduce Chinese text even when the target language is not Chinese, attributed to language-detection interference during code-switching.
- Cleaning extraneous model outputs (e.g., 'The speaker said:') reduced average WER by 6.75% and SAER by 5.34%.
Limitations
- Dataset may be misused for voice synthesis fraud; access is gated by licensing terms.
- Audio quality cannot be assessed by LLM judges, so LLM overall scores are out of 90 rather than 100.
- The evaluation relies on GPT-4o both as generator and judge, which may introduce bias.
Experimental setup
Datasets: MDCC CSLR ASCEND MCE SEAME ArzEn BangorTalk MediaParl
Benchmarks: SwitchLingua
Metrics: WER CER SEM (semantic similarity) SAER human score LLM score
Baselines: Qwen2-Audio-7B-Instruct SenseVoice-Small Seamless-M4T-Large-v2 Wav2Vec2-XLSR-53 Whisper-Large-v3 MDCC CSLR ASCEND MCE SEAME ArzEn BangorTalk
Builds on: Poplack constraints LaBSE AutoGen CAMEL Model Context Protocol
Closest papers by meaning
- Language Without Borders: A Dataset and Benchmark for Code-Switching Lip Reading (NeurIPS 2024 · similarity 0.90)
- WorldSpeech: A Multilingual Speech Corpus from Around the World (NeurIPS 2026 · similarity 0.88)
- WAXAL: A Large-Scale Multilingual African Language Speech Corpus (NeurIPS 2026 · similarity 0.88)
- LAMA-UT: Language Agnostic Multilingual ASR Through Orthography Unification and Language-Specific Transliteration (AAAI 2025 · similarity 0.87)
- MultiTalk: Scaling Full-Duplex Speech Models to Long, Multi-Party, Bilingual Conversation (NeurIPS 2026 · similarity 0.87)
- MMSU: A Massive Multi-task Spoken Language Understanding and Reasoning Benchmark (ICLR 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.