Research map / Label-efficient and robust learning / Recognition and long-tail
Merlin L48 Spectrogram Dataset
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the Merlin L48 Spectrogram dataset, a fine-grained real-world single-positive multi-label bird-sound benchmark with geo and checklist negative-prior regimes, benchmarks SPML methods against a COCO synthetic counterpart, and proposes asset-level prediction consistency regularization.
Problem
Existing single-positive multi-label (SPML) benchmarks are synthetic (random single positives from fully labeled datasets) and do not reflect real-world label distributions or fine-grained confusions; SPML methods are unvalidated on realistic data.
Approach
A real-world SPML bird-spectrogram dataset where each recording has a single expert-annotated target species, plus extended settings with geographic and eBird checklist priors providing confirmed negatives; a regularizer enforces consistency of predictions across clips from the same recording (asset).
Key results
- On L48 target-only, LS achieves 56.4 mAP versus 52.2 for BCE-AN and 66.4 for BCE-Full with asset regularization; asset regularization improves BCE-AN from 52.2 to 56.1.
- On COCO, EM and LL-R reach about 71 mAP while BCE-AN reaches 64.4 and BCE-Full 76.4.
- L48 is about 14 mAP harder than COCO under full supervision (62.4 vs 76.4), and LL variants fall below BCE-AN on L48 target-only.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add asset-level prediction consistency regularization (ablation) | SPML methods · L48 target-only | mAP: 52.2 $\pm$ 0.5 -> 56.1 $\pm$ 1.1 (BCE-AN); 48.0 $\pm$ 0.9 -> 54.1 $\pm$ 0.6 (LL-Ct) | Helped |
| Add asset-level prediction consistency regularization (ablation) | LS · L48 target-only | mAP: 56.4 $\pm$ 0.7 -> 56.4 $\pm$ 0.7 | No clear effect |
| Add asset-level prediction consistency regularization (ablation) | LL-Cp · L48 target-only | mAP: 43.8 $\pm$ 0.8 -> 44.4 $\pm$ 0.6 | No clear effect |
| Use label smoothing instead of BCE-AN | ResNet50 · L48 target-only | mAP: 52.2 $\pm$ 0.5 -> 56.4 $\pm$ 0.7 | Helped |
| Use LL-Ct instead of BCE-AN | ResNet50 · L48 target-only | mAP: 52.2 $\pm$ 0.5 -> 48.0 $\pm$ 0.9 | Hurt |
| Use LL-Cp instead of BCE-AN | ResNet50 · L48 target-only | mAP: 52.2 $\pm$ 0.5 -> 43.8 $\pm$ 0.8 | Hurt |
| Use LL-R instead of BCE-AN | ResNet50 · COCO target-only | mAP: 64.4 $\pm$ 0.1 -> 71.4 $\pm$ 0.1 | Helped |
| Use LL-R instead of BCE-AN | ResNet50 · L48 target-only | mAP: 52.2 $\pm$ 0.5 -> 50.1 $\pm$ 0.8 | Hurt |
| Reveal geographical-prior negatives | SPML methods · L48 | mAP | Mixed |
| Reveal checklist-prior negatives | SPML methods · L48 | mAP | Mixed |
| Reveal geographical-prior negatives | EM · L48 | mAP: 55.3 $\pm$ 1.0 -> 56.3 $\pm$ 0.6 | Helped |
| Reveal checklist-prior negatives | EM · L48 | mAP: 55.3 $\pm$ 1.0 -> 57.2 $\pm$ 0.4 | Helped |
Surprising findings
- Large-loss SPML methods that perform best on COCO (LL-R, LL-Ct, LL-Cp) drop below the BCE-AN baseline on L48, due to fine-grained misclassifications being mistaken for mislabeled negatives.
- Simple label smoothing outperforms most sophisticated SPML methods on L48 target-only.
Limitations
- Covers only 100 bird species in the contiguous US, limiting transfer to other regions and species sets.
- Only 53% of training images with at least one positive are used for training; many images are only partially labeled.
- Gap to fully supervised performance remains 8-10 mAP points even with negative priors and regularization.
Experimental setup
Datasets: Merlin L48 Spectrogram (L48) MS-COCO 2014 Scene-15
Benchmarks: L48 COCO
Metrics: mAP
Baselines: BCE-AN WAN LS ROLE EM LL-R LL-Ct LL-Cp BCE-Full
Builds on: ROLE large-loss methods (LL) entropy maximization (EM) weak assume negative (WAN) label smoothing
Closest papers by meaning
- BirdSet: A Large-Scale Dataset for Audio Classification in Avian Bioacoustics (ICLR 2025 · similarity 0.82)
- Can Class-Priors Help Single-Positive Multi-Label Learning? (NeurIPS 2025 · similarity 0.82)
- Fine-grained Classes and How to Find Them (ICML 2024 · similarity 0.81)
- Semi-supervised Multi-label Learning with Balanced Binary Angular Margin Loss (NeurIPS 2024 · similarity 0.81)
- Fusing Conditional Submodular GAN and Programmatic Weak Supervision (AAAI 2024 · similarity 0.80)
- SSLAM: Enhancing Self-Supervised Models with Audio Mixtures for Polyphonic Soundscapes (ICLR 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.