Research map / Vision-language and multimodal / Human and embodied modeling
EchoWorld: Learning Motion-Aware World Models for Echocardiography Probe Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EchoWorld, a JEPA-style cardiac ultrasound world model pre-trained with spatial masked-anatomy and motion-conditioned prediction tasks, then fine-tuned with a motion-aware attention module for echocardiography probe guidance.
Problem
Probe guidance for echocardiography requires predicting probe movements to reach standard cardiac planes, which demands anatomical knowledge and integration of historical visual-motion data; existing representations and guidance architectures do not capture this well.
Approach
Pre-train a ViT encoder as a JEPA-style world model with masked anatomical prediction plus contrastive motion-conditioned visual prediction, then fine-tune with motion-aware attention that injects pairwise relative probe poses into keys and values across frame tokens.
Key results
- Single-frame protocol: EchoWorld average error 8.15 vs. best pre-trained baseline EchoCLIP 8.37 across ten standard planes.
- Sequential protocol: EchoWorld average error 7.05 vs. Sequence-aware 7.42, Decision Transformer 7.44, US-GuideNet 7.72.
- Ablation: combining spatial and motion world modeling reduces single-frame average error from 9.07 (no pre-training) to 8.15.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EchoWorld instead of EchoCLIP for single-frame guidance | Single-frame guidance · Ten standard planes | Avg MAE: 8.37 -> 8.15 | Helped |
| Use EchoWorld instead of Seq-aware for sequential guidance | Sequential probe guidance · Ten standard planes | Avg MAE: 7.42 -> 7.05 | Helped |
| Add spatial world modeling (ablation) | EchoWorld pre-training · Single-frame protocol, ten standard planes | Avg MAE: 9.07 -> 8.39 | Helped |
| Add motion world modeling (ablation) | EchoWorld pre-training · Single-frame protocol, ten standard planes | Avg MAE: 9.07 -> 8.16 | Helped |
| Use both spatial and motion world modeling (ablation) | EchoWorld pre-training · Single-frame protocol, ten standard planes | Avg MAE: 9.07 -> 8.15 | Helped |
| Add pose information to the guidance model (ablation) | DeiT sequential guidance · Sequential protocol, ten standard planes | Avg MAE: 8.62 -> 7.53 | Helped |
| Add pose information to the guidance model (ablation) | EchoWorld sequential guidance · Sequential protocol, ten standard planes | Avg MAE: 7.98 -> 7.05 | Helped |
Limitations
- Evaluation is offline on recorded expert demonstrations rather than real-person or robot trials.
- Dataset is private and collected from a single clinical source, limiting reproducibility of data.
- Relies on imitation of expert demonstrations and ground-truth movements for fine-tuning.
Experimental setup
Datasets: in-house expert demonstration cardiac ultrasound dataset with synchronized probe poses
Benchmarks: in-house cardiac ultrasound probe guidance dataset (single-frame and sequential protocols)
Metrics: mean absolute error (translation mm, rotation degrees)
Baselines: DeiT DINOv2 BioMedCLIP LVM-Med US-MoCo US-MAE USFM EchoCLIP US-GuideNet Decision Transformer Sequence-aware
Builds on: JEPA I-JEPA MoCo Decision Transformer RCDM
Closest papers by meaning
- EchoXFlow: A Beamspace Echocardiography Dataset for Cardiac Motion, Flow, and Function (NeurIPS 2026 · similarity 0.86)
- U2-BENCH: Benchmarking Large Vision-Language Models on Ultrasound Understanding (ICLR 2026 · similarity 0.84)
- Self-supervised Learning of Echocardiographic Video Representations via Online Cluster Distillation (NeurIPS 2025 · similarity 0.83)
- SonoGym: High Performance Simulation for Challenging Surgical Tasks with Robotic Ultrasound (NeurIPS 2025 · similarity 0.83)
- SurgVista: Long-Horizon Surgical World Modeling with Plausible Instrument-Tissue Dynamics (NeurIPS 2026 · similarity 0.82)
- Bidirectional Recurrence for Cardiac Motion Tracking with Gaussian Process Latent Coding (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.