Research map / Vision-language and multimodal / Cross-modal learning
Audio-Visual Floorplan Reconstruction
Senthil Purushwalkam, Sebastià Vicenc Amengual Garí, Vamsi Krishna Ithapu, Carl Schissler, Philip Robinson, Abhinav Gupta, Kristen Grauman
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add audio (device-generated) to RGB-only (ablation) | AV-Map · Interior map, 4 steps, 40m2 | AP: 69.05 -> 73.67 | Improved |
| add audio (device-generated) to RGB-only (ablation) | AV-Map · Interior map, 4 steps, 40m2 | Accuracy: 64.20 -> 66.51 | Improved |
| add audio (device-generated) to RGB-only (ablation) | AV-Map · Interior map, 4 steps, 40m2 | Edge AP: 52.93 -> 55.21 | Improved |
| add RGB to audio-only (ablation) | AV-Map · Interior map, 4 steps, 40m2 | AP: 70.44 -> 73.67 | Improved |
| add RGB to audio-only (Env. All Room) (ablation) | AV-Map · Interior map, environment-generated All Room | AP: 65.42 -> 71.38 | Improved |
| add environment-generated audio (Env. Telephone) to RGB-only (ablation) | AV-Map · Interior map, 4 steps | AP: 69.05 -> 72.30 | Improved |
| use environment-generated All Room audio instead of device-generated chirp | AV-Map · Interior map, 4 steps | AP: 73.67 -> 71.38 | Hurt |
| train on sequences (t_V=4) instead of single steps (t_V=1) (ablation) | AV-Map · Interior map, dev-generated audio | AP: 68.91 -> 73.61 | Improved |
| train on sequences (t_V=4) instead of single steps (t_V=1) (ablation) | AV-Map · Interior map, dev-generated audio | Accuracy: 62.46 -> 66.51 | Improved |
| add audio to RGB-only at 16 steps with 164m2 area (ablation) | AV-Map · Interior map AP, 164m2 | AP: 76.00 -> 80.71 | Improved |
| use AV-Map RGB-only variant versus OccAnt (no audio) | AV-Map RGB-only · Interior map, 4 steps | AP: 60.27 -> 69.05 | Improved |
Closest papers by meaning
- FloorNet: A Unified Framework for Floorplan Reconstruction from 3D Scans (ECCV 2018 · similarity 0.88)
- SoundSpaces: Audio-Visual Navigation in 3D Environments (ECCV 2020 · similarity 0.87)
- PolyRoom: Room-aware Transformer for Floorplan Reconstruction (ECCV 2024 · similarity 0.87)
- Connecting the Dots: Floorplan Reconstruction Using Two-Level Queries (CVPR 2023 · similarity 0.86)
- CAGE: Continuity-Aware edGE Network Unlocks Robust Floorplan Reconstruction (NeurIPS 2025 · similarity 0.86)
- SAVVY: Spatial Awareness via Audio-Visual LLMs through Seeing and Hearing (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
