Research map / Video / Video-language understanding
Self-supervised Learning of Echocardiographic Video Representations via Online Cluster Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DISCOVR, a dual-branch self-supervised framework for echocardiography video representation that couples a clustering-based video encoder with an online masked image encoder via a semantic cluster distillation loss.
Problem
Self-supervised video representation learning for echocardiography is hard due to high inter-sample similarity, low SNR ultrasound, and the lack of domain-specific pretrained models and labels.
Approach
A video student-teacher (DINO/SWaV-style) encoder with Sinkhorn prototype clustering is trained jointly with an online masked image encoder; a semantic cluster distillation loss aligns video token clusters with image-encoder cluster targets, injecting fine-grained spatial semantics into temporal features without pretrained teachers or heavy augmentation.
Key results
- Zero-shot F1 on EchoNet Dynamic 61.45 vs 57.56 for best baseline (MVD); linear probing F1 77.63 vs 75.50 for SIGMA on EchoNet Dynamic.
- CAMUS segmentation Dice 0.844 with frozen backbone and linear+upsampling head, vs 0.816 for UNet with BYOL pretraining and 0.747 for VideoMAE.
- LVEF prediction MAE 7.79 with linear probing and 6.32 with last-3-block fine-tuning on EchoNet-Dynamic.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add semantic cluster distillation (SCD) to video self-distillation (ablation) | DISCOVR · EchoNet-Dynamic kNN | F1: 48.23 -> 61.45 | Helped |
| Add image self-distillation and SCD alongside video self-distillation (ablation) | DISCOVR · EchoNet-Dynamic kNN | F1: 48.23 -> 61.45 | Helped |
| Use image self-distillation only instead of full DISCOVR (ablation) | DISCOVR · EchoNet-Dynamic kNN | F1: 49.43 -> 61.45 | Hurt |
| Use ViT-Base rather than ViT-Small (ablation) | DISCOVR · EchoNet-Dynamic kNN | F1: 57.52 -> 61.45 | Helped |
| Increase masking ratio from 50% to 90% (ablation) | DISCOVR · EchoNet-Dynamic kNN | F1: 52.98 -> 61.45 | Helped |
| Increase clip length from 16 to 64 frames (ablation) | DISCOVR · EchoNet-Dynamic kNN | F1: 55.68 -> 61.45 | Helped |
| Use DISCOVR instead of MVD | zero-shot classification · EchoNet-Dynamic | F1: 57.56 -> 61.45 | Helped |
| Use DISCOVR instead of SIGMA | linear probing · EchoNet-Dynamic | F1: 75.50 -> 77.63 | Helped |
| Use DISCOVR representations instead of DeepLabV3 with BYOL pretraining | CAMUS segmentation · frozen-backbone segmentation | Dice: 0.819 -> 0.844 | Helped |
| Fine-tune the last three encoder blocks rather than use linear probing | DISCOVR · EchoNet-Dynamic LVEF prediction | MAE: 7.79 -> 6.32 | Helped |
| Use DISCOVR rather than VideoMAE | linear probing · Breast Ultrasound | Balanced Accuracy: 61.75 -> 63.76 | Helped |
| Use DISCOVR rather than VideoMAE | linear probing · DermMNIST | Accuracy: 68.83 -> 71.68 | Helped |
Surprising findings
- Video-only self-distillation alone scores far below the full model (F1 48.23 vs 61.45 on EchoNet Dynamic), and image-only distillation alone is also weak (F1 49.43).
Limitations
- Scope limited to echocardiography; not evaluated on other imaging modalities in the main paper.
- Two fetal datasets are private, limiting reproducibility; limited demographic and geographic diversity.
- No error bars or repeated runs for statistical significance.
Experimental setup
Datasets: FetalEcho1 FetalEcho2 EchoNet Dynamic EchoPediatric LVH RVENET CAMUS Breast Ultrasound Images DermMNIST Kinetics-400
Benchmarks: EchoNet Dynamic EchoPediatric LVH RVENET FetalEcho1 FetalEcho2 CAMUS Breast Ultrasound Images DermMNIST Kinetics-400
Metrics: F1 balanced accuracy accuracy AUC Dice MAE RMSE top-1 accuracy
Baselines: VideoMAE MGMAE SIGMA MVD RAD-DINO MNAD MemAE C2FPL
Builds on: VideoMAE SIGMA DINO Sinkhorn-Knopp clustering
Closest papers by meaning
- MemSAM: Taming Segment Anything Model for Echocardiography Video Segmentation (CVPR 2024 · similarity 0.86)
- EchoXFlow: A Beamspace Echocardiography Dataset for Cardiac Motion, Flow, and Function (NeurIPS 2026 · similarity 0.86)
- U2-BENCH: Benchmarking Large Vision-Language Models on Ultrasound Understanding (ICLR 2026 · similarity 0.85)
- Autoregressive Sequence Modeling for 3D Medical Image Representation (AAAI 2025 · similarity 0.84)
- Revisiting MAE Pre-training for 3D Medical Image Segmentation (CVPR 2025 · similarity 0.84)
- ECHOPulse: ECG Controlled Echocardio-gram Video Generation (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.