Research map / Video / Video segmentation and tracking
EPIC-SOUNDS: A Large-Scale Dataset of Actions That Sound
Jaesung Huh, Jacob Chalk, Evangelos Kazakos, Dima Damen, Andrew Zisserman
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune backbone instead of linear probe (ablation) | SSAST · EPIC-SOUNDS recognition (validation) | top-1 accuracy: 28.74 -> 53.47 | Improved |
| fine-tune backbone instead of linear probe (ablation) | ASF · EPIC-SOUNDS recognition (validation) | top-1 accuracy: 45.53 -> 53.75 | Improved |
| add visual modality (TIM) | ASF audio-only · EPIC-SOUNDS recognition (validation) | top-1 accuracy: 53.75 -> 58.49 | Improved |
| add visual modality (MTCN) | ASF audio-only · EPIC-SOUNDS recognition (entire test) | top-1 accuracy: 54.86 -> 57.96 | Improved |
| add visual modality to ActionFormer | ActionFormer · EPIC-SOUNDS detection (validation) | average mAP: 13.5 -> 14.5 | Improved |
| add visual modality to TriDet | TriDet · EPIC-SOUNDS detection (validation) | average mAP: 13.3 -> 14.8 | Improved |
| add visual modality to TriDet | TriDet · EPIC-SOUNDS detection (detection test) | average mAP: 12.6 -> 13.0 | Improved |
| add visual modality to ActionFormer | ActionFormer · EPIC-SOUNDS detection (entire test) | average mAP: 11.5 -> 12.5 | Improved |
| use TriDet instead of ActionFormer, audio-only | detection · EPIC-SOUNDS detection (validation) | average mAP: 13.5 -> 13.3 | Mixed |
| use ActionFormer-AV instead of TriDet-AV | audio-visual detection · EPIC-SOUNDS detection (detection test) | average mAP: 13.0 -> 13.6 | Mixed |
| fusion weight alpha 0.8 (ActionFormer) / 0.7 (TriDet), audio-dominant (ablation) | audio-visual detection · EPIC-SOUNDS detection | average mAP | Improved |
| human annotators vs ASF on audio-only recognition | audio recognition · EPIC-SOUNDS recognition (validation subset, 412 clips) | accuracy: 20.8% vs 21.1% | No clear effect |
Closest papers by meaning
- TIM: A Time Interval Machine for Audio-Visual Action Recognition (CVPR 2024 · similarity 0.88)
- SoundingActions: Learning How Actions Sound from Narrated Egocentric Videos (CVPR 2024 · similarity 0.86)
- Adapting to the Unknown: Training-Free Audio-Visual Event Perception with Dynamic Thresholds (CVPR 2025 · similarity 0.86)
- Action Dubber: Timing Audible Actions via Inflectional Flow (ICML 2025 · similarity 0.86)
- Towards Open-Vocabulary Audio-Visual Event Localization (CVPR 2025 · similarity 0.86)
- Dynamic Derivation and Elimination: Audio Visual Segmentation with Enhanced Audio Semantics (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
