Research map / Image generation / Autoregressive and latent generation
EchoDiffusion: Waveform Conditioned Diffusion Models for Echo-Based Depth Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EchoDiffusion, a latent diffusion model for echo-based depth estimation that encodes echo spectrograms with a multi-scale MALF-Net and conditions reverse diffusion on Wav2Vec-derived waveform embeddings.
Problem
Echo-based depth estimation typically relies on UNet-style encoder-decoders that struggle to extract fine details from echo waveforms and to perform high-precision multi-scale feature extraction.
Approach
Encode echo spectrograms into a latent space with MALF-Net (ASPP plus ASFF) as the start of a forward diffusion process, and guide the reverse denoising with conditional embeddings derived from raw echo waveforms via a frozen Wav2Vec model (EWDE), then regress depth from the features.
Key results
- On Replica, EchoDiffusion achieves RMSE 0.913 versus 0.921 for the co-attention model and REL 0.604 (slightly worse than co-attention's 0.560), with delta<1.25 of 0.515.
- On Matterport3D, EchoDiffusion achieves RMSE 1.702 versus 1.752 for Bat-Net and REL 0.512 versus 0.569 for Echo-Net.
- Ablation on Replica shows MALF-Net RMSE 0.913 vs. 0.925 for ASPP-only, 0.938 for ASFF-only, and 0.929 for FPN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use EchoDiffusion instead of Co-attention | echo-based depth estimation · Replica test | RMSE: 0.921 -> 0.913 | Helped |
| use EchoDiffusion instead of Co-attention | echo-based depth estimation · Replica test | REL: 0.560 -> 0.604 | Hurt |
| use EchoDiffusion instead of Co-attention | echo-based depth estimation · Replica test | δ < 1.25²: 0.636 -> 0.668 | Helped |
| use EchoDiffusion instead of Bat-Net | echo-based depth estimation · Matterport3D test | RMSE: 1.752 -> 1.702 | Helped |
| use EchoDiffusion instead of Bat-Net | echo-based depth estimation · Matterport3D test | REL: 0.583 -> 0.512 | Helped |
| use EchoDiffusion instead of Echo-Net | echo-based depth estimation · Matterport3D test | δ < 1.25²: 0.642 -> 0.659 | Helped |
| use MALF-Net instead of FPN (ablation) | EchoDiffusion · Replica test | RMSE: 0.929 -> 0.913 | Helped |
| use MALF-Net instead of ASFF without ASPP (ablation) | EchoDiffusion · Replica test | RMSE: 0.938 -> 0.913 | Helped |
| use MALF-Net instead of ASPP without ASFF (ablation) | EchoDiffusion · Replica test | RMSE: 0.925 -> 0.913 | Helped |
| use MALF-Net instead of FPN (ablation) | EchoDiffusion · Matterport3D test | RMSE: 1.741 -> 1.702 | Helped |
| use MALF-Net instead of ASFF without ASPP (ablation) | EchoDiffusion · Matterport3D test | RMSE: 1.722 -> 1.702 | Helped |
| use MALF-Net instead of ASPP without ASFF (ablation) | EchoDiffusion · Matterport3D test | RMSE: 1.742 -> 1.702 | Helped |
Surprising findings
- EchoDiffusion has slightly worse REL than the co-attention baseline on Replica (0.604 vs 0.560), which the authors acknowledge.
Limitations
- Improvements over baselines are modest on several metrics, and REL is worse than the co-attention model on Replica.
- Co-attention baseline is only compared on Replica because its code was not published.
- Baseline Echo-Net was run in single-echo mode by zeroing the image input, which may not reflect its intended multimodal use.
Experimental setup
Datasets: Replica Matterport3D
Benchmarks: Replica Matterport3D
Metrics: RMSE REL log10 delta < 1.25 delta < 1.25^2 delta < 1.25^3
Baselines: Echo-Net Bat-Net Co-attention model
Builds on: Stable Diffusion Wav2Vec ASPP ASFF UNet
Closest papers by meaning
- ECoDepth: Effective Conditioning of Diffusion Models for Monocular Depth Estimation (CVPR 2024 · similarity 0.83)
- A Versatile Diffusion Transformer with Mixture of Noise Levels for Audiovisual Generation (NeurIPS 2024 · similarity 0.83)
- Zero-Shot Novel View and Depth Synthesis with Multi-View Geometric Diffusion (CVPR 2025 · similarity 0.83)
- DepthFM: Fast Generative Monocular Depth Estimation with Flow Matching (AAAI 2025 · similarity 0.83)
- MicroDiffusion: Implicit Representation-Guided Diffusion for 3D Reconstruction from Limited 2D Microscopy Projections (CVPR 2024 · similarity 0.83)
- Likelihood Training of Cascaded Diffusion Models via Hierarchical Volume-preserving Maps (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.