Research map / Architectures and efficiency / RNNs and normalization
Hear You Say You: An Efficient Framework for Marine Mammal Sounds’ Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes XCFSMN, a knowledge distillation framework where a DenseNet121 + TDNN/X-vector + XCA-cFSMN teacher with temperature-cooling KD compresses into a small pruned cFSMN student for marine mammal sound classification.
Problem
Early detection and classification of marine mammals from acoustic recordings is needed, but existing deep CNN and transformer classifiers are computationally expensive and have not been evaluated robustly on noisy, imbalanced data.
Approach
A teacher fusing an X-vector extractor, DenseNet121 and an XCA-based cFSMN is distilled into a much smaller cFSMN student, using a temperature-cooling schedule (starting at T=20, decaying by factor 0.1 on early-stop triggers) to bridge the teacher-student architecture gap; the student also adds instance normalization, DropKey in the XCA layer, and ~30% global unstructured pruning.
Key results
- Student model achieves F1 0.910 on Merged-5 vs 0.828 for CC-FSMN and 0.925 recall-level teacher F1 0.986 on MobySound-5 (teacher), with the student at 0.973.
- Student model has 3.15M parameters versus 81.3M for the teacher, and matches CC-FSMN inference time (0.3 s on Merged-5) while CNN and transformer baselines take up to 4 to 17 s.
- Temperature cooling (F1 0.910) outperforms fixed temperature T=10 (F1 0.854) and a warming schedule (F1 0.887) on the merged dataset.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add InstanceNorm after cFSMN (ablation) | Student model · Merged-5 | F1: 0.828 -> 0.839 | Helped |
| replace Dropout with DropKey (ablation) | Student model · Merged-5 | F1: 0.828 -> 0.844 | Helped |
| add InstanceNorm and replace Dropout with DropKey (ablation) | Student model · Merged-5 | F1: 0.828 -> 0.850 | Helped |
| add temperature-cooling knowledge distillation (ablation) | Student model · Merged-5 | F1: 0.850 -> 0.910 | Helped |
| use fixed KD temperature T=10 instead of temperature cooling (ablation) | Student model · Merged-5 | F1: 0.854 vs 0.910 | Hurt |
| increase KD temperature during training instead of cooling it (ablation) | Student model · Merged-5 | F1: 0.887 vs 0.910 | Hurt |
| distill knowledge from the teacher using temperature cooling | Student model · WMMSD-12 | F1: 0.787 -> 0.820 | Helped |
| distill knowledge from the teacher using temperature cooling | Student model · WMMSD-5 | F1: 0.646 -> 0.791 | Helped |
| distill knowledge from the teacher using temperature cooling | Student model · MobySound-5 | F1: 0.922 -> 0.973 | Helped |
| distill knowledge from the teacher using temperature cooling | Student model · Merged-5 | F1: 0.828 -> 0.910 | Helped |
Surprising findings
- Transformers (DeiT III, Longformer, FNet) perform poorly on this small acoustic dataset, with FNet F1 as low as 0.120 on WMMSD-12.
- The authors claim acoustic classification outperforms prior image-based marine mammal classification, though the comparison uses different datasets and metrics.
Limitations
- Class imbalance is severe (e.g., Minke whale 0.05 hours in WMMSD), and the teacher-student gap leaves the student below the teacher's F1.
- Evaluation is limited to two datasets and five shared whale classes for the main comparison.
- No hyperparameter or code details were released; the temperature schedule is heuristic.
Experimental setup
Datasets: WMMSD MobySound
Benchmarks: WMMSD-12 WMMSD-5 MobySound-5 Merged-5
Metrics: precision recall macro F1 training time inference time TOPSIS ranking
Baselines: CC-FSMN AlexNet ResNet18 ResNet50 VGG16 DenseNet DeiT III Longformer FNet
Builds on: CC-FSMN cFSMN X-vector DropKey Knowledge distillation (Mirzadeh et al. 2020)
Closest papers by meaning
- OpenWhistle: A Large-Scale Longitudinal Dataset and Benchmark of Bottlenose Dolphin Vocalizations (NeurIPS 2026 · similarity 0.83)
- NatureLM-audio: an Audio-Language Foundation Model for Bioacoustics (ICLR 2025 · similarity 0.82)
- BirdSet: A Large-Scale Dataset for Audio Classification in Avian Bioacoustics (ICLR 2025 · similarity 0.82)
- Towards A Translative Model of Sperm Whale Vocalization (NeurIPS 2025 · similarity 0.82)
- Fantastic Animals and Where to Find Them: Segment Any Marine Animal with Dual SAM (CVPR 2024 · similarity 0.82)
- Audio Scanning Network: Bridging Time and Frequency Domains for Audio Classification (AAAI 2024 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.