Research map / Image generation / Fast diffusion sampling
Towards Precise Embodied Dialogue Localization via Causality Guided Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CGD, a causality-guided diffusion model that regresses 2D coordinates for embodied dialogue localization, using back-door and front-door adjustments to remove confounders in the denoising condition.
Problem
Embodied dialogue localization on top-down maps is hard to do precisely with heatmap-based encoder-decoder methods, which depend heavily on resolution and generalize poorly to unseen environments due to dataset biases.
Approach
Model coordinates directly with a diffusion denoiser conditioned on visual and text tokens, and inject causal de-confounded features (back-door adjustment for observable confounders, front-door adjustment via a VQ-VAE mediator for unobservable ones) as classifier-free-style guidance during denoising.
Key results
- Single-shot CGD reaches 16.72% Acc0 and 69.56% Acc5 on valUnseen, improving over DiaLoc by 9.7 points on Acc0 and 29.15 points on Acc5 per the text.
- Multi-shot CGD achieves 11.76% Acc0 and 62.88% Acc5 on valUnseen, surpassing DiaLoc by 3.32 and 15.73 points.
- Ablation shows diffusion alone and either BDA or FDA individually improve over the MLP baseline, with the full combination performing best.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use diffusion coordinate regression instead of the non-diffusion MLP (ablation) | CGD · WAY valUnseen | Acc0: 4.17 -> 8.94 | Helped |
| Add BDA to diffusion coordinate regression (ablation) | CGD · WAY valUnseen | Acc0: 8.94 -> 10.11 | Helped |
| Add FDA to diffusion coordinate regression (ablation) | CGD · WAY valUnseen | Acc0: 8.94 -> 12.98 | Helped |
| Add FDA to diffusion + BDA (ablation) | CGD · WAY valUnseen | Acc0: 10.11 -> 16.72 | Helped |
| Add BDA to diffusion + FDA (ablation) | CGD · WAY valUnseen | Acc0: 12.98 -> 16.72 | Helped |
| Use diffusion instead of MLP while retaining BDA and FDA (ablation) | CGD · WAY valUnseen | Acc0: 8.67 -> 16.72 | Helped |
| Use 20 rather than 0 KDE kernels (ablation) | CGD · WAY valUnseen | Acc0: 11.10 -> 12.62 | Helped |
| Increase KDE kernels from 20 to 50 (ablation) | CGD · WAY valUnseen | Acc0: 12.62 -> 13.47 | Helped |
| Increase KDE kernels from 50 to 80 (ablation) | CGD · WAY valUnseen | Acc0: 13.47 -> 14.87 | Helped |
| Increase KDE kernels from 80 to 100 (ablation) | CGD · WAY valUnseen | Acc0: 14.87 -> 16.72 | Helped |
| Use CGD instead of DiaLoc† | single-shot localization · WAY valUnseen | Acc0: 7.02 -> 16.72 | Helped |
| Use CGD instead of DiaLoc† | single-shot localization · WAY valUnseen | Acc5: 40.41 -> 69.56 | Helped |
Limitations
- Inference is relatively slow due to iterative denoising.
- Evaluated only on the WAY dataset with floor-level evaluation.
Experimental setup
Datasets: WAY dataset
Benchmarks: WAY dataset valSeen WAY dataset valUnseen
Metrics: Acc0 Acc5 (geodesic distance accuracy at 0 m and 5 m)
Baselines: LingUNet DiaLoc LED-BERT
Builds on: DDPM classifier-free guidance DiaLoc LingUNet VQ-VAE
Closest papers by meaning
- DiaLoc: An Iterative Approach to Embodied Dialog Localization (CVPR 2024 · similarity 0.88)
- ERGeoBench: A Comprehensive Benchmark for Embodied Reasoning and Geo-localization in Multimodal Large Language Models (ICML 2026 · similarity 0.86)
- Masked Diffusion Vision-Language Models for Temporal Action Localization (NeurIPS 2026 · similarity 0.85)
- NavBench: Probing Multimodal Large Language Models for Embodied Navigation (NeurIPS 2025 · similarity 0.85)
- I Think, Therefore I Diffuse: Enabling Multimodal In-Context Reasoning in Diffusion Models (ICML 2025 · similarity 0.85)
- Pre-trained Text-to-Image Diffusion Models Are Versatile Representation Learners for Control (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.