Research map / 3D vision / 3D generation
Deep Neural Room Acoustics Primitive
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Learns a self-supervised neural room acoustics field (DeepNeRAP) from two cooperative probing agents' sound recordings, predicting neural RIRs for arbitrary source-receiver positions without RIR ground truth.
Problem
Estimating room impulse responses (RIRs) for arbitrary source and receiver positions in 3D rooms, where measured or supervised RIR data are scarce and costly.
Approach
Two cooperative agents actively probe a room by emitting and receiving sine sweeps; a physics-guided network (grid feature map, multi-scale position-aware features, symmetric add for reciprocity, real/imag time-frequency RIR decoder with ISTFT) is trained only by matching predicted reverberant sound to recorded sound.
Key results
- On Matterport3D, DeepNeRAP reaches SDR 6.62 vs. 5.16 for NAF (best baseline) and f-MSE 1.68 vs. 4.09 for NAF.
- On MeshRIR, DeepNeRAP achieves SDR 7.31 and PSNR 20.15 against NAF's 5.36 and 16.19.
- Under training-sample reduction and noise, DeepNeRAP retains higher SDR than all baselines in Fig. 5 (e.g., 7.13 SDR with 3000 samples vs. 5.32 for NAF).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DeepNeRAP instead of NAF | RIR estimation · Matterport3D | SDR: 5.16˘ 0.09 -> 6.62˘ 0.12 | Helped |
| Use DeepNeRAP instead of NAF | RIR estimation · Matterport3D | t-MSE: 1.01˘ 0.27 -> 0.93 ˘ 0.34 | Helped |
| Use DeepNeRAP instead of NAF | RIR estimation · Matterport3D | PSEQ: 1.40 ˘ 0.41 -> 1.53 ˘ 0.41 | Helped |
| Use DeepNeRAP instead of NAF | RIR estimation · MeshRIR | SDR: 5.36˘ 0.12 -> 7.31˘ 0.10 | Helped |
| Use DeepNeRAP instead of NAF | RIR estimation · MeshRIR | t-MSE: 2.21˘ 0.17 -> 1.13 ˘ 0.20 | Helped |
| Use DeepNeRAP instead of NAF | RIR estimation · MeshRIR | PSEQ: 1.54 ˘ 0.32 -> 1.77 ˘ 0.30 | Helped |
| remove learnable room acoustic grid features (ablation) | DeepNeRAP · 17DRP5sb8fy and MeshRIR | overall RIR evaluation | Hurt |
| remove multi-scale feature aggregation (ablation) | DeepNeRAP · 17DRP5sb8fy and MeshRIR | overall RIR evaluation | Hurt |
| use single-resolution rather than multi-resolution RIR prediction (ablation) | DeepNeRAP · 17DRP5sb8fy and MeshRIR | overall RIR evaluation | Hurt |
| remove position encoding (ablation) | DeepNeRAP · 17DRP5sb8fy and MeshRIR | overall RIR evaluation | Hurt |
Limitations
- Assumes the two probing agents can reach most of the traversable area within a step budget; efficient exploration in real settings is not addressed.
- No existing methods share the exact problem setting, so baselines were modified adaptations.
- Evaluated only on one small real-world room and a synthetic simulator dataset; PESQ acknowledged as imperfect.
Experimental setup
Datasets: SoundSpaces 2.0 / Matterport3D (synthetic, 54 scenes) MeshRIR S32-M441 VCTK anechoic speech
Benchmarks: Matterport3D (SoundSpaces 2.0) probing dataset MeshRIR S32-M441
Metrics: t-MSE SDR T60 error f-MSE PSNR SSIM PESQ
Baselines: NAF IR-MLP S2IR-GAN
Builds on: NAF implicit neural representations Neural RIR interpolation (IR-MLP)
Closest papers by meaning
- Hearing Anywhere in Any Environment (CVPR 2025 · similarity 0.90)
- Resounding Acoustic Fields with Reciprocity (NeurIPS 2025 · similarity 0.88)
- Real Acoustic Fields: An Audio-Visual Room Acoustics Dataset and Benchmark (CVPR 2024 · similarity 0.87)
- Hearing Anything Anywhere (CVPR 2024 · similarity 0.87)
- NeRAF: 3D Scene Infused Neural Radiance and Acoustic Fields (ICLR 2025 · similarity 0.86)
- AV-RIR: Audio-Visual Room Impulse Response Estimation (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.