Research map / 3D vision / Depth and stereo
Camera Pose Estimation and Localization with Active Audio Sensing
Karren Yang, Michael Firman, Eric Brachmann, Clément Godard
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add depth-cubemap distillation pretraining to the audio encoder (ablation) | audio-only relative pose · Replica relative pose (standard baseline, <90 deg) | Acc@20deg (%): 21.9 -> 38.1 | Improved |
| add depth-cubemap distillation pretraining to the audio encoder (ablation) | audio-only relative pose · Matterport3D relative pose (standard baseline, <90 deg) | Acc@20deg (%): 13.8 -> 17.7 | Improved |
| add depth-cubemap distillation pretraining to the audio encoder (ablation) | audio descriptor · Replica place recognition (all queries) | Recall@1: 0.59 -> 0.65 | Improved |
| add depth-cubemap distillation pretraining to the audio encoder (ablation) | audio descriptor · Matterport3D place recognition (all queries) | Recall@1: 0.41 -> 0.49 | Improved |
| add depth-cubemap distillation pretraining to the audio encoder (ablation) | audio-visual PoseNet · Matterport3D absolute pose regression (all queries) | position error: 3.34 -> 1.86 | Improved |
| add audio expert to SuperGlue visual matching | visual matching · Replica relative pose (wide baseline, >90 deg) | Acc@20deg (%): 2.0 -> 31.5 | Improved |
| add audio expert to SuperGlue visual matching | visual matching · Replica relative pose (standard baseline, <90 deg) | Acc@20deg (%): 39.9 -> 47.7 | Improved |
| add learned gating instead of fixed gating on visual matching (ablation) | visual matching + audio · Replica relative pose (standard baseline, <90 deg) | Acc@20deg (%): 47.7 -> 55.4 | Improved |
| add learned gating instead of fixed gating on visual matching (ablation) | visual matching + audio · Replica relative pose (wide baseline, >90 deg) | Acc@20deg (%): 31.5 -> 37.6 | Improved |
| add audio features to Siamese visual regression | visual regression · Replica relative pose (standard baseline, <90 deg) | Acc@20deg (%): 21.0 -> 24.1 | Mixed |
| add audio expert and learned gating to NetVLAD descriptors | NetVLAD · Replica place recognition (all queries) | Recall@1: 0.59 -> 0.71 | Improved |
| add audio features to PoseNet | PoseNet · Matterport3D absolute pose regression (all queries) | position error: 2.17 -> 1.86 | Improved |
Closest papers by meaning
- Audio-Visual Camera Pose Estimation with Passive Scene Sounds and In-the-Wild Video (ECCV 2026 · similarity 0.89)
- Sound Localization from Motion: Jointly Learning Sound Direction and Camera Rotation (ICCV 2023 · similarity 0.86)
- Supervising Sound Localization by In-the-wild Egomotion (CVPR 2025 · similarity 0.86)
- Prompting Segmentation with Sound Is Generalizable Audio-Visual Source Localizer (AAAI 2024 · similarity 0.85)
- Object-aware Sound Source Localization via Audio-Visual Scene Understanding (CVPR 2025 · similarity 0.84)
- Robust Audio-Visual Segmentation via Audio-Guided Visual Convergent Alignment (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
