Research map / Video / Video segmentation and tracking
SoundNet: Learning Sound Representations from Unlabeled Video
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| student-teacher transfer of visual knowledge (KL loss) into the sound network trained on unlabeled video, vs. training from scratch on labeled data (ablation) | 8-layer SoundNet · ESC-50 | accuracy: 51.1% -> 72.9% | Helped |
| student-teacher transfer from unlabeled video vs. training from scratch on labeled data (ablation) | 8-layer SoundNet · ESC-10 | accuracy: 75.5% -> 92.2% | Helped |
| KL divergence loss instead of l2 loss on teacher outputs (ablation) | SoundNet 8 layer · ESC-50 | accuracy: 47.8% -> 72.9% | Helped |
| KL divergence loss instead of l2 loss on teacher outputs (ablation) | SoundNet 8 layer · ESC-10 | accuracy: 81.5% -> 92.2% | Helped |
| using both ImageNet and Places teachers instead of one (ablation) | SoundNet 8 layer · ESC-50 | accuracy: 69.5% (ImageNet) / 71.1% (Places) -> 72.9% | Helped |
| using both ImageNet and Places teachers instead of one (ablation) | SoundNet 8 layer · ESC-10 | accuracy: 89.8% / 89.5% -> 92.2% | Helped |
| increasing depth from 5 to 8 layers, with visual transfer from unlabeled video (ablation) | SoundNet · ESC-50 | accuracy: 66.1% -> 72.9% | Helped |
| increasing depth from 5 to 8 layers, with visual transfer from unlabeled video (ablation) | SoundNet · ESC-10 | accuracy: 86.8% -> 92.2% | Helped |
| increasing depth from 5 to 8 layers without unlabeled video (scratch on labeled data only) (ablation) | SoundNet · ESC-50 | accuracy: 65.0% -> 51.1% | Hurt |
| pool5 features instead of conv7 (layer choice for features) (ablation) | SoundNet 8 layer (AlexNet teacher) · ESC-50 | accuracy: 63.8% (conv7) -> 74.2% (pool5) | Helped |
| VGG teacher instead of AlexNet teacher (ablation) | SoundNet 8 layer features on DCASE · DCASE | accuracy: 84% -> 88% (pool5) | Helped |
| VGG teacher instead of AlexNet teacher (ablation) | SoundNet 8 layer features on ESC-50 · ESC-50 | accuracy: 74.2% -> 72.9% (pool5) | Hurt |
Closest papers by meaning
- Learning to Localize Sound Source in Visual Scenes (CVPR 2018 · similarity 0.89)
- From Vision to Audio and Beyond: A Unified Model for Audio-Visual Representation and Generation (ICML 2024 · similarity 0.89)
- Separating the "Chirp" from the "Chat": Self-supervised Visual Grounding of Sound and Language (CVPR 2024 · similarity 0.88)
- Seeing Speech and Sound: Distinguishing and Locating Audio Sources in Visual Scenes (CVPR 2025 · similarity 0.87)
- Few-Shot Incremental Learning via Foreground Aggregation and Knowledge Transfer for Audio-Visual Semantic Segmentation (AAAI 2025 · similarity 0.86)
- CAV-MAE Sync: Improving Contrastive Audio-Visual Mask Autoencoders via Fine-Grained Alignment (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.