Research map / Large language models / Language, safety and interpretability
WolBanking77: Wolof Banking Speech Intent Classification Dataset
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces WolBanking77, a Wolof banking intent classification dataset (9,791 text sentences across 77 intents plus a 263-utterance, ~4h audio subset of 10 intents) and benchmarks text and speech models on it.
Problem
Intent classification and spoken language understanding resources are scarce for low-resource, oral African languages such as Wolof, especially in banking, where high illiteracy limits access to text-based services.
Approach
Translate the Banking77 text queries into Wolof (with French) through linguists, pair a smaller MINDS-14-based set with multi-speaker Wolof audio recordings, and benchmark text classifiers (BERT-family, SLMs, LASER3 pipelines) and ASR models (Canary-1b-flash, Phi-4-multimodal, Distil-whisper) on the resulting data.
Key results
- AfroXLMR reaches about 79% weighted F1 on the 5k-sample text split in few-shot/fine-tuning settings, the best among multilingual encoders.
- Canary-1b-flash achieves 0.59% WER on the 4-hour audio set, versus 3.1% for Phi-4-multimodal-instruct and 4.63% for Distil-whisper-large-v3.5.
- Llama-3.2-3B-Instruct reaches F1 0.75 on the 5k split and 0.55 on the full set, while LASER3+MLP reaches 0.55 F1 on 5k.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| tune CNN hyperparameters (ablation) | LASER3+CNN · WolBanking77 5k samples | F1: 0.01 -> 0.49 | Helped |
| tune CNN hyperparameters (ablation) | LASER3+CNN · WolBanking77 all samples | F1: 0.01 -> 0.32 | Helped |
| use Llama-3.2-3B-Instruct instead of Llama-3.2-1B-Instruct | Llama-3.2 · WolBanking77 5k samples | F1: 0.72 -> 0.75 | Helped |
| use Llama-3.2-3B-Instruct instead of Llama-3.2-1B-Instruct | Llama-3.2 · WolBanking77 all samples | F1: 0.46 -> 0.55 | Helped |
| fine-tune Canary-1b-flash on WolBanking77 audio | Canary-1b-flash · WolBanking77 audio, 4 hours of speech | WER | Helped |
Surprising findings
- Pretraining on Wolof (AfroLM, Afro-xlmr) does not guarantee good zero-shot or few-shot performance, which the authors attribute to dataset difficulty.
- LASER3+CNN without tuning collapses to about 1% F1, while tuning raises it to 49% on 5k samples.
Limitations
- Class imbalance across intents (most frequent 200 vs least 24 samples) may bias models toward majority classes.
- Audio set is small (263 sentences, 10 intents, ~4 hours) and recorded in a controlled environment with possible background noise.
- Spelling errors in rare words noted in ASR output; no language model correction applied.
Experimental setup
Datasets: Banking77 MINDS-14 WolBanking77
Benchmarks: WolBanking77
Metrics: F1 precision recall word error rate
Baselines: BERT-base AfroXLMR AfroLM mDeBERTa-v3-base AfriTeVa V2 Llama-3.2-1B-Instruct Llama-3.2-3B-Instruct Canary-1b-flash Phi-4-multimodal-instruct Distil-whisper-large-v3.5 LASER3+MLP BoW+LR
Builds on: Banking77 MINDS-14 LASER3 Canary Distil-Whisper
Closest papers by meaning
- WAXAL: A Large-Scale Multilingual African Language Speech Corpus (NeurIPS 2026 · similarity 0.83)
- Towards Multi-Intent Spoken Language Understanding via Hierarchical Attention and Optimal Transport (AAAI 2024 · similarity 0.82)
- SwitchLingua: The First Large-Scale Multilingual and Multi-Ethnic Code-Switching Dataset (NeurIPS 2025 · similarity 0.81)
- BIGOS V2 Benchmark for Polish ASR: Curated Datasets and Tools for Reproducible Evaluation (NeurIPS 2024 · similarity 0.81)
- MIntRec2.0: A Large-scale Benchmark Dataset for Multimodal Intent Recognition and Out-of-scope Detection in Conversations (ICLR 2024 · similarity 0.81)
- Objective Soups: Multilingual Multi-Task Modeling for Speech Processing (NeurIPS 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.