Research map / Label-efficient and robust learning / Semi-supervised learning
VQ-Seg: Vector-Quantized Token Perturbation for Semi-Supervised Medical Image Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VQ-Seg, a semi-supervised medical segmentation framework that replaces feature dropout with a codebook-index shuffling perturbation (QPM) in a VQ space, plus a dual-branch reconstruction/segmentation design and DINOv2-guided alignment (PFA).
Problem
Feature-level dropout used for consistency regularization in semi-supervised medical image segmentation is highly sensitive to the dropout rate, with high rates causing collapse and low rates giving negligible regularization.
Approach
Perturb features in a discrete vector-quantized codebook space by stochastically replacing codewords according to distance-based transition probabilities controlled by a single strength epsilon (QPM), with a shared post-quantization space used for both reconstruction and segmentation, and a DINOv2-based contrastive alignment of quantized features.
Key results
- On LC with 5% labels, VQ-Seg reaches Dice 0.6643 and Jaccard 0.5257, versus 0.6493 for the second-best Unimatch.
- On LC with 10% labels, VQ-Seg reaches Dice 0.7852 and HD95 11.6179, outperforming the best baselines.
- On ACDC with 5% labels, VQ-Seg reaches Dice 0.9057, above Unimatch (0.8915).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add QPM to the VQ-embedded Unimatch baseline (ablation) | VQ-Seg · LC 10% labeled | Dice: 0.7443 -> 0.7701 | Helped |
| Add the dual-branch architecture with shared Post-VQ space (ablation) | VQ-Seg + QPM · LC 10% labeled | Dice: 0.7701 -> 0.7784 | Helped |
| Add PFA without the dual-branch architecture (ablation) | VQ-Seg + QPM · LC 10% labeled | Dice: 0.7701 -> 0.7761 | Helped |
| Combine QPM, DB, and PFA (ablation) | VQ-Seg · LC 10% labeled | Dice: 0.7784 -> 0.7852 | Helped |
| Increase perturbation strength ε from 0.3 to 0.7; higher values beyond 0.7 lower performance (ablation) | VQ-Seg · LC 10% labeled | Dice: 0.7741 -> 0.7852; 0.7852 -> 0.7418 | Mixed |
| Vary alignment loss weight λa from 1 to 5 to 10 (ablation) | VQ-Seg · LC 10% labeled | Dice: 0.7720 -> 0.7852 -> 0.7765 | Mixed |
| Vary unlabeled loss weight λu from 1 to 5 to 10 (ablation) | VQ-Seg · LC 10% labeled | Dice: 0.7852 -> 0.7670 -> 0.7843 | Mixed |
| Replace DINOv2 with CLIP, BiomedCLIP, MAE, or Rad-DINO (ablation) | VQ-Seg · LC 5% and 10% labeled | Dice: 0.6643 vs 0.6421; 0.7852 vs 0.7483 | Hurt |
| Increase codebook size from 1024 to 16384 (ablation) | VQ-Seg · LC 5% and 10% labeled | Dice: 0.6531 -> 0.6643; 0.7608 -> 0.7852 | Helped |
| Increase codebook size beyond 16384 to 32768 or 65536 (ablation) | VQ-Seg · LC 5% and 10% labeled | Dice: 0.6643 -> 0.6595 -> 0.6415; 0.7852 -> 0.7764 -> 0.7638 | Hurt |
| Use VQ-Seg instead of the strongest reported baseline | VQ-Seg · LC 5% labeled | Dice: 0.6493 -> 0.6643 | Helped |
| Use VQ-Seg instead of the strongest reported baseline | VQ-Seg · LC 10% labeled | Dice: 0.7555 -> 0.7852 | Helped |
Surprising findings
- Dropout rates of 0.7 or higher sharply degrade segmentation, with DR=0.9 yielding unusable predictions in a fully supervised setting.
- Enlarging the codebook beyond 16,384 (to 32,768 or 65,536) slightly degrades Dice, attributed to lower code utilization.
Limitations
- The perturbation operates only in discrete VQ space and is hard to extend to continuous feature representations.
- Using a foundation model adds computational overhead.
- Experiments are limited to 2D slices and two datasets, one of which (LC) is private.
Experimental setup
Datasets: LC (Lung Cancer) dataset ACDC
Benchmarks: LC dataset ACDC
Metrics: Dice Jaccard HD95 ASD
Baselines: UNet nnUNet UA-MT MCNet SSNet BCP ARCO ABD Unimatch
Builds on: Unimatch DINOv2 Straight-Through Estimator MC-Dropout
Closest papers by meaning
- GapMatch: Bridging Instance and Model Perturbations for Enhanced Semi-Supervised Medical Image Segmentation (AAAI 2025 · similarity 0.89)
- PathVQ: Reforming Computational Pathology Foundation Model for Whole Slide Image Analysis via Vector Quantization (NeurIPS 2025 · similarity 0.85)
- Enhancing SAM with Efficient Prompting and Preference Optimization for Semi-supervised Medical Image Segmentation (CVPR 2025 · similarity 0.85)
- Semi-supervised TEE Segmentation via Interacting with SAM Equipped with Noise-Resilient Prompting (AAAI 2024 · similarity 0.85)
- Steady Progress Beats Stagnation: Mutual Aid of Foundation and Conventional Models in Mixed Domain Semi-Supervised Medical Image Segmentation (CVPR 2025 · similarity 0.84)
- Towards the Uncharted: Density-Descending Feature Perturbation for Semi-supervised Semantic Segmentation (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.