Research map / Video / Video segmentation and tracking
STARSS23: An Audio-Visual Dataset of Spatial Recordings of Real Scenes with Spatiotemporal Annotations of Sound Events
Kazuki Shimada, Archontis Politis, Parthasaarathy Sudarsanam, Daniel Aleksander Krause, Kengo Uchida, Sharath Adavanne, Aapo Hakala, Yuichiro Koyama, Naoya Takahashi, Shusuke Takahashi, Tuomas Virtanen, Yuki Mitsufuji
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add visual person bounding-box embedding to audio-only SELD (13 classes, FOA) | CRNN multi-ACCDOA SELD · STARSS23 dev-set-test 13 classes FOA | F-score 20deg: 14.8 -> 13.2 | Hurt |
| add visual person bounding-box embedding (13 classes, FOA) | CRNN multi-ACCDOA SELD · STARSS23 dev-set-test 13 classes FOA | localization error: 55.9 -> 51.6 | Improved |
| add visual person bounding-box embedding (13 classes, MIC) | CRNN multi-ACCDOA SELD · STARSS23 dev-set-test 13 classes MIC | localization error: 66.9 -> 60.0 | Improved |
| add visual person bounding-box embedding (13 classes, MIC) | CRNN multi-ACCDOA SELD · STARSS23 dev-set-test 13 classes MIC | F-score 20deg: 10.7 -> 10.0 | No clear effect |
| add visual person bounding-box embedding on body-related classes only (FOA) | CRNN multi-ACCDOA SELD · STARSS23 5 body-related classes FOA | F-score 20deg: 18.0 -> 22.4 | Improved |
| add visual person bounding-box embedding on body-related classes only (MIC) | CRNN multi-ACCDOA SELD · STARSS23 5 body-related classes MIC | F-score 20deg: 16.0 -> 20.9 | Improved |
| add visual person bounding-box embedding on body-related classes only (MIC) | CRNN multi-ACCDOA SELD · STARSS23 5 body-related classes MIC | localization error: 58.4 -> 28.9 | Improved |
| add person-box visual input, evaluated on non-body-related classes | CRNN multi-ACCDOA SELD · STARSS23 13 classes FOA, non-body classes | F-score 20deg | Hurt |
| use CNN-based visual features instead of object-detector bounding boxes (ablation) | audio-visual SELD · STARSS23 | SELD performance | Hurt |
| use FOA instead of MIC audio format | SELD systems · STARSS23 dev-set-test 13 classes | F-score 20deg: 10.0 -> 13.2 (audio-visual) | Improved |
Closest papers by meaning
- MRSAudio: A Large-Scale Multimodal Recorded Spatial Audio Dataset with Refined Annotations (NeurIPS 2025 · similarity 0.89)
- Towards Audio-Visual Navigation in Noisy Environments: A Large-Scale Benchmark Dataset and an Architecture Considering Multiple Sound-Sources (AAAI 2025 · similarity 0.88)
- Adapting to the Unknown: Training-Free Audio-Visual Event Perception with Dynamic Thresholds (CVPR 2025 · similarity 0.87)
- SAVVY: Spatial Awareness via Audio-Visual LLMs through Seeing and Hearing (NeurIPS 2025 · similarity 0.87)
- Object-aware Sound Source Localization via Audio-Visual Scene Understanding (CVPR 2025 · similarity 0.87)
- Hear, Localize, and Reason: Spatially Aware Scene Understanding for Audio-visual LLMs (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
