Research map / 3D vision / 3D generation
Hearing Anywhere in Any Environment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes XRIR, a cross-room RIR predictor that fuses panorama-depth geometry features with a few reference RIRs via attention-weighted spectrogram combination, and introduces the ACOUSTICROOMS simulated dataset (260 rooms, 300K RIRs).
Problem
Predicting room impulse responses for new rooms with unseen geometry and surface materials, where existing neural RIR methods must be retrained per room with dense measurements.
Approach
Combine a panorama-depth-based reflection geometry encoder (ViT) with a ResNet-18 encoder of a few reference RIRs, and predict the target RIR as an attention-weighted sum of reference log-magnitude spectrograms, enabling generalization to unseen rooms without per-room training.
Key results
- On unseen ACOUSTICROOMS splits, XRIR (K=8) reaches T60 error of 10.53% and C50 error of 1.457 dB versus 11.64% and 2.667 dB for Nearest Neighbor (K=8).
- On four real-world rooms, XRIR (K=8) outperforms Diff-RIR (K=12) on EDT, C50 and T60 in all rooms except T60 where Nearest Neighbor is better.
- XRIR (K=8) achieves seen-split EDT error 0.038 s versus 0.064 s for Nearest Neighbor.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use XRIR (K=8) instead of Nearest Neighbor (K=8) | Cross-room RIR prediction · ACOUSTICROOMS seen split | EDT error (s): 0.064 -> 0.038 | Helped |
| Use XRIR (K=8) instead of Nearest Neighbor (K=8) | Cross-room RIR prediction · ACOUSTICROOMS seen split | C50 error (dB): 1.717 -> 0.940 | Helped |
| Use XRIR (K=8) instead of Nearest Neighbor (K=8) | Cross-room RIR prediction · ACOUSTICROOMS seen split | T60 error (%): 8.94 -> 8.13 | Helped |
| Use XRIR (K=8) instead of Nearest Neighbor (K=8) | Cross-room RIR prediction · ACOUSTICROOMS unseen split | EDT error (s): 0.090 -> 0.055 | Helped |
| Use XRIR (K=8) instead of Nearest Neighbor (K=8) | Cross-room RIR prediction · ACOUSTICROOMS unseen split | C50 error (dB): 2.667 -> 1.457 | Helped |
| Use XRIR (K=8) instead of Nearest Neighbor (K=8) | Cross-room RIR prediction · ACOUSTICROOMS unseen split | T60 error (%): 11.64 -> 10.53 | Helped |
| Use XRIR (K=8) instead of Nearest Neighbor (K=8) | Sim-to-real RIR prediction · Classroom | EDT error: 0.108 -> 0.093 | Helped |
| Use XRIR (K=8) instead of Nearest Neighbor (K=8) | Sim-to-real RIR prediction · Classroom | T60 error: 2.71 -> 6.25 | Hurt |
| Use XRIR (K=8) instead of Nearest Neighbor (K=8) | Sim-to-real RIR prediction · Dampened Room | EDT error: 0.044 -> 0.044 | No clear effect |
| Use XRIR (K=8) instead of Nearest Neighbor (K=8) | Sim-to-real RIR prediction · Hallway | C50 error: 0.990 -> 0.954 | Helped |
| Use XRIR (K=8) instead of Nearest Neighbor (K=8) | Sim-to-real RIR prediction · Hallway | T60 error: 3.02 -> 3.20 | Hurt |
| Use XRIR (K=8) instead of Nearest Neighbor (K=8) | Sim-to-real RIR prediction · Complex Room | EDT error: 0.091 -> 0.077 | Helped |
Surprising findings
- Few-Shot RIR performs poorly on the ACOUSTICROOMS data, attributed to its UNet decoder and co-located binaural echo design.
- XRIR underperforms the Nearest Neighbor baseline on T60 error in all four real rooms, attributed to sensitivity of T60 to measurement noise.
Limitations
- Trained on simulation with higher SNR than real measurements, so struggles with low-SNR real recordings.
- Diff-RIR requires per-room training and is computationally infeasible for large complex rooms, so it is excluded from simulated comparisons.
- Predicts magnitude spectrograms converted with Griffin-Lim, which loses phase information.
Experimental setup
Datasets: ACOUSTICROOMS Hearing-Anything-Anywhere Dataset
Benchmarks: ACOUSTICROOMS (seen and unseen splits) Hearing-Anything-Anywhere real-world rooms
Metrics: EDT error C50 error T60 error
Baselines: Random Across Rooms Random Same Room Nearest Neighbor Linear Interpolation Few-Shot RIR Diff-RIR
Builds on: INRAS Few-Shot RIR Diff-RIR ResNet-18 Vision Transformer
Closest papers by meaning
- Hearing Anything Anywhere (CVPR 2024 · similarity 0.92)
- Deep Neural Room Acoustics Primitive (ICML 2024 · similarity 0.90)
- AV-RIR: Audio-Visual Room Impulse Response Estimation (CVPR 2024 · similarity 0.88)
- Real Acoustic Fields: An Audio-Visual Room Acoustics Dataset and Benchmark (CVPR 2024 · similarity 0.87)
- Acoustic Volume Rendering for Neural Impulse Response Fields (NeurIPS 2024 · similarity 0.86)
- Resounding Acoustic Fields with Reciprocity (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.