Research map / Label-efficient and robust learning / Recognition and long-tail
The iNaturalist Sounds Dataset
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces iNatSounds, a ~230K-recording, 5,500-species weakly labeled bioacoustic dataset from iNaturalist, benchmarks CNN/ViT backbones with multiclass vs multilabel (assume-negative plus mean-teacher) objectives, and evaluates transfer to strongly labeled bird and frog datasets.
Problem
Large-scale, fine-grained, easily accessible species sound datasets are lacking; existing audio datasets are coarse, licensing-restricted, or small in species coverage, and weakly labeled bioacoustic data needs suitable training objectives.
Approach
Build a large, freely licensed, species-labeled animal sound dataset from iNaturalist with a year-based train/val/test split, treat spectrograms as images for standard vision backbones, and compare multiclass against multilabel single-positive learning (assume-negative loss with mean-teacher pseudo-labeling), showing multilabel transfers better to strongly labeled downstream data.
Key results
- Best iNatSounds test model (ViT-B-16 with geo-prior, multilabel + mean teacher) reaches 65.4 mAP on iNatSounds test-set-level validation metrics; multiclass ResNet-50 with geo-prior reaches 60.7 Top-1 on test.
- Multilabel with mean teacher improves downstream mAP, e.g., MobileNet-V3 on Powdermill rises from 32.9 to 37.1 and on SWAMP from 29.9 to 33.9.
- BirdNET outperforms the iNatSounds models on SSW60 (86.7 Top-1) and Powdermill, though the authors note downstream data may be in BirdNET's training set.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply the SINR geo-prior to multiclass predictions | iNatSounds multiclass models · iNatSounds test | Top-1 accuracy: 49.1$$0.17 -> 56.9 $$ 0.19; 52.6 $$ 0.95 -> 60.7 $$ 0.62; 53.6 $$ 0.22 -> 60.3 $$ 0.27 | Helped |
| use an assume-negative multilabel objective instead of multiclass training | iNatSounds models · iNatSounds test | Top-1 accuracy: 49.1$^{0.2}$ -> 42.4$^{0.4}$; 52.6$^{1.0}$ -> 48.1$^{0.4}$; 53.6$^{0.2}$ -> 52.4$^{0.1}$ | Hurt |
| use an assume-negative multilabel objective instead of multiclass training | iNatSounds models · Powdermill and SWAMP | mAP: 24.1$^{0.3}$ -> 39.7$^{1.2}$; 25.0$^{2.4}$ -> 31.6$^{1.2}$; 24.6$^{1.3}$ -> 40.2$^{0.9}$; 24.3$^{2.2}$ -> 39.0$^{0.9}$ | Helped |
| add mean-teacher training to multilabel training | MobileNet-V3 and ResNet-50 · SSW60 | Top-1 accuracy: 61.4$^{1.3}$ -> 68.4$^{1.2}$; 63.8$^{1.4}$ -> 70.4$^{0.9}$ | Helped |
| add mean-teacher training to multilabel training | ViT · Powdermill | mAP: 40.2$^{0.9}$ -> 37.8$^{0.5}$ | Hurt |
| add mixup augmentation (ablation) | multiclass models · iNatSounds test | mAP: 51.6$^{0.2}$ -> 60.0$^{0.3}$; 55.5$^{0.4}$ -> 62.8$^{0.7}$; 56.1$^{1.5}$ -> 64.4$^{0.3}$ | Helped |
| add mixup augmentation (ablation) | multiclass models · Powdermill | mAP: 22.3$^{0.8}$ -> 33.5$^{1.1}$; 24.1$^{0.3}$ -> 36.5$^{1.0}$; 24.6$^{1.3}$ -> 31.7$^{0.2}$ | Helped |
| pretrain on iNatSounds rather than ImageNet | MobileNet-V3 on BirdCLEF 2024 · BirdCLEF 2024 | Class-average Top1: 09.08 -> 12.18 | Helped |
Surprising findings
- Models with the best iNatSounds accuracy are not always the best on downstream strongly labeled datasets; for ResNet-50 and ViT the correlation does not hold.
- MobileNet-V3 with geo-prior (small model) outperforms ViT-B-16 without it on iNatSounds test top-1 (56.9 vs 53.6).
Limitations
- Labels are weak (single species per file, no timestamps), limiting training and evaluation precision.
- Geographic bias toward North America and Europe.
- Potential privacy concerns and misuse risks such as poaching.
Experimental setup
Datasets: iNatSounds SSW60 Powdermill SWAMP AnuraSet BEANS BirdCLEF 2024 iNat2021
Benchmarks: iNatSounds SSW60 Powdermill SWAMP AnuraSet BEANS BirdCLEF 2024
Metrics: Top-1 accuracy Top-5 accuracy mAP mF1 ROC-AUC
Baselines: BirdNET Merlin Sound ID BEANS baselines (SVM, rn50p, rn152p, vggish) ImageNet-pretrained initialization
Builds on: SINR mixup mean teacher ImageNet pretraining BEANS
Closest papers by meaning
- NatureLM-audio: an Audio-Language Foundation Model for Bioacoustics (ICLR 2025 · similarity 0.89)
- OpenWhistle: A Large-Scale Longitudinal Dataset and Benchmark of Bottlenose Dolphin Vocalizations (NeurIPS 2026 · similarity 0.85)
- BirdSet: A Large-Scale Dataset for Audio Classification in Avian Bioacoustics (ICLR 2025 · similarity 0.85)
- RealBirdID: Benchmarking Bird Species Identification in the Era of MLLMs (CVPR 2026 · similarity 0.83)
- MammAlps: A Multi-view Video Behavior Monitoring Dataset of Wild Mammals in the Swiss Alps (CVPR 2025 · similarity 0.83)
- Combining Observational Data and Language for Species Range Estimation (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.