Research map / Label-efficient and robust learning / Semi-supervised learning
Combinatorial CNN-Transformer Learning with Manifold Constraints for Semi-supervised Medical Image Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes M-CnT, a dual-student CNN-Transformer semi-supervised medical segmentation framework with intra-student class-wise manifold losses and inter-student covariance-based knowledge transfer.
Problem
Semi-supervised medical image segmentation suffers from CNN limits in global context and from neglected class-level distribution structure; CNN and Transformer students learn heterogeneous features that are hard to combine with few labels.
Approach
Combines a U-Net (CNN) and Swin-UNet (Transformer) as two students, applies a prototype-based intra-student class-wise aggregation and separation loss in a manifold space, and aligns the two students' class-wise covariance matrices via Frobenius distance, plus an entropy-based selection of pseudo labels from the more confident student.
Key results
- On ACDC with 3% labels, M-CnT reaches DSC 0.753 vs 0.704 for the runner-up CTCT.
- On ISIC with 3% labels, DSC 0.779 vs 0.760 for the next best (DCT/CTCT-level baselines), with HD reduced by 4.2mm over the runner-up.
- On Synapse with 15% labels, DSC 0.653 vs 0.604 for CTCT, a +4.9% improvement.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add pixel-level consistency loss Lpixel (ablation) | M-CnT · ACDC and ISIC, 3% labeled | DSC: ACDC: 0.510 -> 0.715; ISIC: 0.677 -> 0.734 | Helped |
| Add Laggregate to compact each class cluster (ablation) | M-CnT · ACDC and ISIC, 3% labeled | DSC: ACDC: +1.1%; ISIC: +1.5% | Helped |
| Add Leliminate to increase distance among classes (ablation) | M-CnT · ACDC and ISIC, 3% labeled | DSC: ACDC: +2.3%; ISIC: +2.5% | Helped |
| Add both Laggregate and Leliminate (ablation) | M-CnT · ACDC and ISIC, 3% labeled | DSC: ACDC: +2.8%; ISIC: +3.4% | Helped |
| Add inter-student knowledge transfer loss Linter (ablation) | M-CnT · ACDC and ISIC, 3% labeled | DSC | Helped |
| Use manifold constraints instead of ED, IFVD, or PLCL class-wise losses (ablation) | M-CnT · ACDC and ISIC, 3% labeled | DSC | Helped |
| Use Gaussian warming-up λ(t) rather than a fixed λm (ablation) | M-CnT · ACDC, 3% labeled | DSC | Helped |
| Set λp=1 (ablation) | M-CnT · ACDC, 3% labeled | DSC | Helped |
| Apply manifold constraints at the final decoder stage (ablation) | M-CnT · ACDC and ISIC, 3% labeled | DSC: ACDC: +3.8%; ISIC: +4.5% | Helped |
Limitations
- Improvements over baselines are evaluated only on three medical datasets with 2D slices and crops.
- Hyper-parameter sensitivity to the warming-up weight and loss weights is shown only on ACDC with 3% labels.
Experimental setup
Datasets: ACDC ISIC Synapse
Benchmarks: ACDC ISIC Synapse
Metrics: DSC HD
Baselines: MT UA-MT EM DCT CCT CPS ICT DAN URPC CTCT SSNet ICT-Med
Builds on: Mean Teacher Swin-UNet CTCT multi-manifold attention
Closest papers by meaning
- Semi-supervised Medical Image Segmentation through Dual-task Consistency (AAAI 2021 · similarity 0.90)
- SemiCVT: Semi-Supervised Convolutional Vision Transformer for Semantic Segmentation (CVPR 2023 · similarity 0.89)
- nnWNet: Rethinking the Use of Transformers in Biomedical Image Segmentation and Calling for a Unified Evaluation Benchmark (CVPR 2025 · similarity 0.86)
- Semi-supervised TEE Segmentation via Interacting with SAM Equipped with Noise-Resilient Prompting (AAAI 2024 · similarity 0.86)
- SGTC: Semantic-Guided Triplet Co-training for Sparsely Annotated Semi-Supervised Medical Image Segmentation (AAAI 2025 · similarity 0.86)
- GraphCL: Graph-based Clustering for Semi-Supervised Medical Image Segmentation (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.