Research map / Label-efficient and robust learning / Recognition and long-tail
BirdSet: A Large-Scale Dataset for Audio Classification in Avian Bioacoustics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces BirdSet, a large-scale avian bioacoustics audio classification dataset collection (6,800+ training hours from Xeno-Canto, 400+ hours of strongly labeled soundscape test data) and benchmarks six deep models under three multi-label training scenarios.
Problem
Audio classification lacks large-scale, accessible, standardized benchmarks beyond AudioSet, and avian bioacoustics in particular lacks curated focal training data paired with strongly labeled soundscape test sets for multi-label evaluation under covariate and task shift.
Approach
Curates a Hugging Face dataset collection that pairs large focal-recording training sets from Xeno-Canto with eight strongly labeled soundscape test datasets, uniform eBird label formatting, and a standardized multi-label benchmark protocol (cmAP, AUROC, T1-Acc) across large-training, medium-training, and dedicated fine-tuning scenarios.
Key results
- ConvNext consistently outperforms other models across test datasets in the large-training scenario on AUROC, often exceeding Perch.
- Perch excels at top-1 accuracy (T1-Acc) despite its simpler EfficientNet architecture.
- Large-scale training (LT and MT) generally improves performance over dedicated fine-tuning (DT) with AST and W2V2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use large-scale or medium-scale supervised training instead of dedicated fine-tuning | BirdSet multi-label classification models · BirdSet soundscape evaluation datasets | cmAP, AUROC, and T1-Acc | Helped |
| Fine-tune general-audio pre-trained AST or W2V2 without domain-specific training | AST and W2V2 · BirdSet dedicated fine-tuning (DT) | overall benchmark performance | Hurt |
| Use ConvNext instead of other evaluated models | BirdSet large training (LT) · BirdSet test datasets | AUROC | Helped |
| Use ConvNext instead of Perch | BirdSet large training (LT) · BirdSet test datasets | AUROC | Helped |
| Use more complex models such as AST or ConvNext rather than smaller EfficientNet models | BirdSet large training (LT) · BirdSet test datasets | classification performance | Helped |
| Use waveform transformers EAT or W2V2 instead of spectrogram-based models | BirdSet multi-label classification · BirdSet test datasets | classification performance | Hurt |
Surprising findings
- Fine-tuning general-domain pre-trained models (AST, W2V2) on dedicated subsets is less effective than large-scale bird-domain training.
- Waveform transformers EAT and W2V2 are competitive but slightly inferior to spectrogram-based models.
Limitations
- Does not analyze underlying dataset characteristics affecting model robustness.
- Does not include the most recent audio transformers or a thorough study of pre-training influence.
Experimental setup
Datasets: XCL XCM VOX POW
Benchmarks: PER NES UHH HSN NBP SSW SNE
Metrics: cmAP AUROC T1-Acc
Baselines: Perch BirdNET
Builds on: AudioSet Perch multi-label mixup
Closest papers by meaning
- NatureLM-audio: an Audio-Language Foundation Model for Bioacoustics (ICLR 2025 · similarity 0.89)
- The iNaturalist Sounds Dataset (NeurIPS 2024 · similarity 0.85)
- Massive Sound Embedding Benchmark (MSEB) (NeurIPS 2025 · similarity 0.84)
- Vocal Call Locator Benchmark (VCL) for localizing rodent vocalizations from multi-channel audio (NeurIPS 2024 · similarity 0.84)
- OpenWhistle: A Large-Scale Longitudinal Dataset and Benchmark of Bottlenose Dolphin Vocalizations (NeurIPS 2026 · similarity 0.84)
- Synthio: Augmenting Small-Scale Audio Classification Datasets with Synthetic Data (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.