Research map / Detection and segmentation / Open-set and few-shot detection
Pancakes: Consistent Multi-Protocol Image Segmentation Across Biomedical Domains
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Pancakes predicts a distribution over segmentation protocols for an unseen biomedical image set and samples multiple label maps whose label indices are consistent across images.
Problem
Existing biomedical segmentation models either support only one fixed protocol or require manual prompting/examples to specify the protocol, and universal models produce label maps whose indices are inconsistent across images of the same set.
Approach
A UNet predicts distribution parameters per pixel; a shallow conv head conditioned on sinusoidal embeddings of a sampled (protocol count, label count) pair produces multiple complete multi-label protocols, trained with a min-over-protocols-and-labels Dice loss averaged over image sets so each label index refers to the same structure across images.
Key results
- Pancakes outperforms baselines on all seven held-out datasets, often by more than 20 Dice points on Set Dice.
- At set size 1, Pancakes performs similarly to SAM and is superior to other baselines; baselines degrade as set size grows while Pancakes stays steady.
- Using Pancakes predictions to initialize ScribblePrompt reduces required clicks by about half (3-4 instead of 5-8).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Pancakes instead of foundation segmentation baselines | Pancakes · seven held-out datasets | Set Dice | Helped |
| Increase set size beyond 1 | Pancakes and baselines · held-out datasets | Set Dice | Mixed |
| Increase the number of protocols M (ablation) | Pancakes · held-out datasets | Set Dice | Helped |
| Vary the maximum labels K per protocol (ablation) | Pancakes · held-out datasets | Set Dice | Mixed |
| Train with real and synthetic data instead of either alone (ablation) | Pancakes · held-out datasets, M=16 | Dice | Helped |
| Train with real and synthetic data instead of either alone (ablation) | Pancakes · held-out datasets, M=8 | Dice | No clear effect |
| Initialize ScribblePrompt with Pancakes predictions | ScribblePrompt · held-out datasets, set sizes larger than one | Dice: +5 Dice points | Helped |
| Initialize ScribblePrompt with Pancakes predictions | ScribblePrompt · held-out datasets | number of required interactions: by half | Helped |
Surprising findings
- Universal models that fully parcellate images produce label indices that are semantically inconsistent across images, so label correspondence is hard to recover.
- Synthetic Anatomix data improved Dice only for M=16 with statistical significance; for M=8 the change was not significant.
Limitations
- Evaluation relies on Dice overlap with only one available ground-truth label per dataset, which only partially assesses plausibility of multiple protocols.
- Requires users to choose M and K a priori and map labels to known structures.
- Works on 2D images resized to 128x128 and has not been evaluated for societal biases.
Experimental setup
Datasets: Megamedical Anatomix (synthetic, from TotalSegmentator) ACDC PanDental SpineWeb
Benchmarks: QUBIQ Prostate SCD WBC BUID LIDC-IDRI DDTI STARE HipXRay
Metrics: Set Dice Set IoU Set Surface Distance runtime
Baselines: SAM ScribblePrompt MedSAM UnSAM
Builds on: Anatomix Megamedical Segment Anything Model Mask2Former
Closest papers by meaning
- S2S2: Semantic Stacking for Robust Semantic Segmentation in Medical Imaging (AAAI 2025 · similarity 0.87)
- Constructing and Exploring Intermediate Domains in Mixed Domain Semi-supervised Medical Image Segmentation (CVPR 2024 · similarity 0.86)
- Versatile Medical Image Segmentation Learned from Multi-Source Datasets via Model Self-Disambiguation (CVPR 2024 · similarity 0.86)
- Steady Progress Beats Stagnation: Mutual Aid of Foundation and Conventional Models in Mixed Domain Semi-Supervised Medical Image Segmentation (CVPR 2025 · similarity 0.86)
- Why Invariance is Not Enough for Biomedical Domain Generalization and How to Fix It (NeurIPS 2026 · similarity 0.86)
- GapMatch: Bridging Instance and Model Perturbations for Enhanced Semi-Supervised Medical Image Segmentation (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.