Research map / 3D vision / 3D generation
Dense 2D-3D Indoor Prediction with Sound via Aligned Cross-Modal Distillation
Heeseung Yun, Joonil Na, Gunhee Kim
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SAM spatial-alignment distillation instead of direct feature distance | U-Net student · DAPS-Depth | MAE: up to 10% MAE reduction | Improved |
| direct feature-level distillation (Rank/MTA) added to pseudo-GT loss | U-Net and DPT students · DAPS-Depth | MAE | No clear effect |
| use time-band (W'x1) patches instead of 16x16 patches (ablation) | DPT+SAM · DAPS-Depth | MAE: 0.8903 -> 0.8497 | Improved |
| use frequency-band (1xH') patches instead of 16x16 patches (ablation) | DPT+SAM · DAPS-Depth | MAE: 0.8903 -> 0.8902 | No clear effect |
| use spatially varying embeddings instead of non-spatial embeddings (ablation) | DPT+SAM · DAPS-Depth | MAE: 0.8777 -> 0.8497 | Improved |
| use binaural instead of mono audio (ablation) | DPT+SAM · DAPS-Depth | MAE: 1.0783 -> 0.8497 | Improved |
| replace learnable embeddings with oracle visual features (ablation) | DPT+SAM · DAPS-Depth | MAE: 0.8497 -> 0.5622 | Improved |
| align low-level layers (SAM_1,2) with audio features (ablation) | DPT+SAM · DAPS-Depth | accuracy | No clear effect |
| reduce number of spatial embeddings per layer to one (ablation) | DPT+SAM · DAPS-Depth | accuracy | Hurt |
| add SAM to U-Net instead of Pseudo-GT only | U-Net student · DAPS-Semantic | mIoU: 0.320 -> 0.363 | Improved |
| add MTA distillation to pseudo-GT | U-Net student · DAPS-Semantic | mIoU: 0.320 -> 0.316 | No clear effect |
| add SAM to U-Net for 3D reconstruction | U-Net student · DAPS-3D | IoU: 0.150 (MSE) -> 0.178 | Improved |
Closest papers by meaning
- OWL : Geometry-Aware Spatial Reasoning for Audio Large Language Models (ICLR 2026 · similarity 0.86)
- Deep Neural Room Acoustics Primitive (ICML 2024 · similarity 0.85)
- Separating the "Chirp" from the "Chat": Self-supervised Visual Grounding of Sound and Language (CVPR 2024 · similarity 0.85)
- Seeing Speech and Sound: Distinguishing and Locating Audio Sources in Visual Scenes (CVPR 2025 · similarity 0.85)
- Hear, Localize, and Reason: Spatially Aware Scene Understanding for Audio-visual LLMs (NeurIPS 2026 · similarity 0.85)
- SoundVista: Novel-View Ambient Sound Synthesis via Visual-Acoustic Binding (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
