Research map / Video / Video segmentation and tracking
FocusMAE: Gallbladder Cancer Detection from Ultrasound Videos with Focused Masked Autoencoders
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FocusMAE, a video masked autoencoder that biases masking toward detected high-information regions (via RPN-generated region priors), and applies it to video-based gallbladder cancer detection from ultrasound plus COVID CT detection.
Problem
Automated gallbladder cancer (GBC) detection from ultrasound relies on single frames that generalize poorly; random masking in video MAEs wastes reconstruction effort on low-information background since malignant regions occupy a small portion of frames.
Approach
Inflate masking probabilities of tokens whose centers fall inside object-localization region priors (from a Faster R-CNN RPN trained on GBCU) within an adaptive MAE sampler trained with an AdaMAE-style reconstruction-error loss, so masking concentrates on high-information regions.
Key results
- FocusMAE reaches 96.4% accuracy on GBC video detection (5-fold CV) with 100% sensitivity, versus 94.7% for AdaMAE and 84% for GBCNet/RadFormer.
- On COVID-CT-MD, FocusMAE achieves 0.885 accuracy vs 0.852 for VideoMAE.
- Ablation: 95% masking ratio gives the best accuracy (96.4%); MSE reconstruction loss gives 2.1% higher mean accuracy than L1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use video-based FocusMAE instead of image-based GBCNet | GBC detection · US video GBC detection | accuracy: 0.840 -> 0.964 | Helped |
| use FocusMAE instead of AdaMAE | US video GBC detection · US video GBC detection | accuracy: 0.947±0.053 -> 0.964±0.047 | Helped |
| use FocusMAE instead of VideoMAE | COVID detection · COVID-CT-MD | accuracy: 0.852 -> 0.885 | Helped |
| bias masking toward candidate high-information regions rather than random masking | MAE pretraining for US video GBC detection · US video GBC detection | accuracy | Helped |
| increase masking ratio to 95% (ablation) | FocusMAE · US video GBC detection | accuracy: 96.4% | Helped |
| use MSE instead of L1 reconstruction loss (ablation) | FocusMAE pretraining · US video GBC detection | accuracy: 2.1% | Helped |
| use a larger ViT backbone instead of ViT-S (ablation) | FocusMAE · US video GBC detection | accuracy | Hurt |
| increase decoder depth from 4 to 6 (ablation) | FocusMAE · US video GBC detection | accuracy | Helped |
| increase decoder depth from 6 to 8 (ablation) | FocusMAE · US video GBC detection | accuracy | Hurt |
| increase decoder depth beyond 8 (ablation) | FocusMAE · US video GBC detection | accuracy | Helped |
Surprising findings
- AdaMAE did not perform significantly better than VideoMAE in the authors' experiments, and larger ViT backbones (ViT-B) underperformed ViT-S, suggesting overfitting.
- Increasing decoder depth beyond a shallow network improved accuracy, contrary to the usual shallow MAE decoder design.
Limitations
- Small dataset of 91 videos from limited patients, with 5-fold CV results showing high variance (SD up to ~0.05).
- Region priors depend on a separately trained detector and biopsy-proven labels for added videos; motion-based priors are not applicable to stationary organs.
- Generality is shown on only one additional task (COVID CT).
Experimental setup
Datasets: GBUSV GBCU COVID-CT-MD Kinetics-400
Benchmarks: Curated GBC US video dataset (5-fold CV) COVID-CT-MD
Metrics: accuracy specificity sensitivity
Baselines: GBCNet RadFormer AdaMAE VideoMAEv2 VideoMAE Video-Swin TimeSformer VidTr ViT DeiT PVTv2 ResNet50
Builds on: VideoMAE AdaMAE Masked Autoencoders TimeSformer
Closest papers by meaning
- FOCUS: Knowledge-enhanced Adaptive Visual Compression for Few-shot Whole Slide Image Classification (CVPR 2025 · similarity 0.81)
- CSC-PA: Cross-image Semantic Correlation via Prototype Attentions for Single-network Semi-supervised Breast Tumor Segmentation (CVPR 2025 · similarity 0.81)
- Self-supervised Learning of Echocardiographic Video Representations via Online Cluster Distillation (NeurIPS 2025 · similarity 0.80)
- Transformer-Based Video-Structure Multi-Instance Learning for Whole Slide Image Classification (AAAI 2024 · similarity 0.80)
- Rethinking Transformer for Long Contextual Histopathology Whole Slide Image Analysis (NeurIPS 2024 · similarity 0.80)
- Gastric-X: A Multimodal Multi-Phase Benchmark Dataset for Advancing Vision-Language Models in Gastric Cancer Analysis (CVPR 2026 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.