Research map / Video / Video segmentation and tracking
MemSAM: Taming Segment Anything Model for Echocardiography Video Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adapts SAM (via SAMUS) to semi-supervised echocardiography video segmentation using a space-time memory prompt and a memory reinforcement module that reweights memory features by predicted masks.
Problem
Echocardiography video segmentation suffers from speckle noise, ambiguous boundaries, and large inter-frame shape/scale changes, and annotations are sparse (only ED/ES frames), while SAM-based methods handle only single images or need per-frame prompts.
Approach
A memory prompt derived from an XMem-style space-time memory (sensory, working, long-term) replaces per-frame external prompts, feeding temporally-aware high-level features into the SAM mask decoder; a memory reinforcement step weights image embeddings with predicted probability maps before memory update to suppress speckle noise.
Key results
- On CAMUS-Semi, MemSAM reaches 93.31 mDice and 87.61 mIoU, versus 91.11 mDice for the SAMUS baseline.
- On EchoNet-Dynamic, MemSAM reaches 92.78 mDice versus 91.79 for SAMUS and 91.36 for UNet.
- Removing memory prompts drops CAMUS-Semi performance back to the SAMUS baseline (91.11 mDice); LVEF correlation on CAMUS-Semi improves to 78.92% from 67.55% for SAMUS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the initial point prompt (ablation) | MemSAM · CAMUS-Semi | mDice: 93.17 -> 93.31 | No clear effect |
| add memory prompting (ablation) | MemSAM · CAMUS-Semi | mDice: 91.11 -> 93.31 | Helped |
| add memory reinforcement (ablation) | MemSAM · CAMUS-Semi | mDice: 92.86 -> 93.31 | Helped |
| use different numbers of initial point prompts (1, 2, 3, 5, or 10) (ablation) | MemSAM · CAMUS-Semi | model performance | No clear effect |
| use MemSAM instead of H2Former | CAMUS-Semi segmentation · CAMUS-Semi | mDice: 91.31 -> 93.31 | Helped |
| use MemSAM instead of SAMUS | EchoNet-Dynamic segmentation · EchoNet-Dynamic | mDice: 91.79 -> 92.78 | Helped |
Surprising findings
- Removing point prompts causes only a minor decline (93.17 vs 93.31 mDice), since the left ventricle location is relatively fixed.
- Under full supervision the gain over semi-supervised training is marginal, and the method still outperforms competitors in both settings.
Limitations
- Failure when the initial frame is extremely poor, causing the whole sequence to be mis-segmented.
- Computational cost is not reduced; the authors note future work on lightweighting.
- Evaluated only on two echocardiography datasets.
Experimental setup
Datasets: CAMUS EchoNet-Dynamic
Benchmarks: CAMUS-Semi CAMUS-Full EchoNet-Dynamic
Metrics: mDice mIoU HD95 ASSD LVEF correlation LVEF bias LVEF std
Baselines: UNet SwinUNet H2Former MedSAM MSA SAMed SonoSAM SAMUS
Builds on: Segment Anything Model SAMUS XMem STM
Closest papers by meaning
- Self-supervised Learning of Echocardiographic Video Representations via Online Cluster Distillation (NeurIPS 2025 · similarity 0.86)
- EchoONE: Segmenting Multiple Echocardiography Planes in One Model (CVPR 2025 · similarity 0.85)
- SAM 2: Segment Anything in Images and Videos (ICLR 2025 · similarity 0.84)
- Enhancing SAM with Efficient Prompting and Preference Optimization for Semi-supervised Medical Image Segmentation (CVPR 2025 · similarity 0.83)
- TSAM: Temporal SAM Augmented with Multimodal Prompts for Referring Audio-Visual Segmentation (CVPR 2025 · similarity 0.83)
- RMem: Restricted Memory Banks Improve Video Object Segmentation (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.