Research map / Image generation / Super-resolution
Semantic Object Prediction and Spatial Sound Super-Resolution with Binaural Sounds
Arun Balajee Vasudevan, Dengxin Dai, Luc Van Gool
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use binaural instead of mono sound input | auditory semantic prediction network · AuditoryTestPseudo (All classes) | mIoU: 22.12 -> 32.01 | Improved |
| add depth prediction as auxiliary task (ablation) | Ours(B) · AuditoryTestPseudo (All classes) | mIoU: 32.01 -> 38.95 | Improved |
| add S3R as auxiliary task (ablation) | Ours(B) · AuditoryTestPseudo (All classes) | mIoU: 32.01 -> 42.64 | Improved |
| add both S3R and depth auxiliary tasks (ablation) | Ours(B) · AuditoryTestManual (All classes) | mIoU: 28.02 -> 38.01 | Improved |
| use all 8 sound channels instead of mono input (ablation) | auditory semantic prediction (no auxiliary tasks) · AuditoryTestPseudo (All classes) | mIoU: 22.12% -> 40.32% | Improved |
| add ASPP module to audio encoder (ablation) | Mono sound audio encoder · AuditoryTestPseudo (All classes) | mIoU: 13.21 -> 22.12 | Improved |
| increase number of S3R output microphone pairs from 1 to 3 (ablation) | Ours(B:S) and Ours(B:SD) · AuditoryTestPseudo (All classes) | mIoU | Improved |
| use input pair (3,8) aligned with camera front rather than other orientations (ablation) | Ours(B) binaural input · AuditoryTestPseudo (All classes) | mIoU | Improved |
| use binaural instead of mono sound | auditory depth prediction · Auditory depth prediction | Abs Rel: 118.88 -> 108.59 | Improved |
| add S3R joint task (ablation) | binaural depth prediction · Auditory depth prediction | Abs Rel: 108.59 -> 90.43 | Improved |
| add semantic joint task (ablation) | binaural depth prediction · Auditory depth prediction | Abs Rel: 108.59 -> 87.96 | Improved |
| add both semantic and S3R joint tasks (ablation) | binaural depth prediction · Auditory depth prediction | Abs Rel: 108.59 -> 84.24 | Improved |
Closest papers by meaning
- Hear, Localize, and Reason: Spatially Aware Scene Understanding for Audio-visual LLMs (NeurIPS 2026 · similarity 0.88)
- DeepASA: An Object-Oriented Multi-Purpose Network for Auditory Scene Analysis (NeurIPS 2025 · similarity 0.88)
- OWL : Geometry-Aware Spatial Reasoning for Audio Large Language Models (ICLR 2026 · similarity 0.87)
- The World is Not Mono: Enabling Spatial Understanding in Large Audio-Language Models (NeurIPS 2026 · similarity 0.87)
- Cyclic Learning for Binaural Audio Generation and Localization (CVPR 2024 · similarity 0.87)
- Prompting Segmentation with Sound Is Generalizable Audio-Visual Source Localizer (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
