Research map / Vision-language and multimodal / Cross-modal learning
Cross-Modal Interactive Perception Network with Mamba for Lung Tumor Segmentation in PET-CT Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces PCLT20K, a large public PET-CT lung tumor segmentation dataset (21,930 slice pairs, 605 patients), and CIPA, a Mamba-based dual-branch network with channel-wise rectification and dynamic cross-modality interaction modules.
Problem
Lung tumor segmentation in PET-CT is limited by small, private datasets and by difficulty fusing complementary PET and CT information (noise, low resolution, complex morphology).
Approach
A fully state-space-model dual-branch network: a channel state space block (CRM) rectifies channels across PET and CT to suppress modality-specific noise, and DCIM uses PET regional tokens (via a region Mamba block) as position cues added to CT local tokens processed by a local Mamba block.
Key results
- CIPA reaches 63.81% IoU, 77.91% F1, 89.01% accuracy and 17.74 HD95 on PCLT20K-TE, best among 12 compared methods (best baseline Sigma 63.26% IoU).
- On the public STS dataset, CIPA achieves 60.33% IoU vs 59.02% for GeminiFusion.
- Ablation: adding CRM and DCIM raises IoU from 62.12% to 63.81%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CIPA instead of Sigma | PCLT20K-TE segmentation · PCLT20K-TE | IoU (%): 63.26 -> 63.81 | Helped |
| Use CIPA instead of GeminiFusion | STS segmentation · STS test set | IoU (%): 59.02 -> 60.33 | Helped |
| Add CRM to the shared Mamba baseline (ablation) | CIPA · PCLT20K-TE | IoU: 62.12 -> 62.70 | Helped |
| Add DCIM to the shared Mamba baseline (ablation) | CIPA · PCLT20K-TE | IoU: 62.12 -> 63.48 | Helped |
| Add CRM and DCIM (ablation) | CIPA · PCLT20K-TE | IoU: 62.12 -> 63.81 | Helped |
| Use 2×2 instead of 4×4 local patches per region (ablation) | DCIM · PCLT20K-TE | IoU: 63.81 -> 62.87 | Hurt |
| Use 8×8 instead of 4×4 local patches per region (ablation) | DCIM · PCLT20K-TE | IoU: 63.81 -> 63.07 | Hurt |
| Apply region and local Mamba blocks to PET rather than the default PET-region/CT-local assignment (ablation) | DCIM · PCLT20K-TE | IoU: 63.81 -> 42.06 | Hurt |
| Apply region and local Mamba blocks to CT rather than the default PET-region/CT-local assignment (ablation) | DCIM · PCLT20K-TE | IoU: 63.81 -> 61.37 | Hurt |
| Apply region Mamba to CT and local Mamba to PET rather than the default assignment (ablation) | DCIM · PCLT20K-TE | IoU: 63.81 -> 60.63 | Hurt |
| Remove region Mamba from PET and use direct PET features (ablation) | DCIM · PCLT20K-TE | IoU: 63.81 -> 62.65 | Hurt |
| Remove local Mamba from CT and use direct CT features (ablation) | DCIM · PCLT20K-TE | IoU: 63.81 -> 62.26 | Hurt |
Surprising findings
- Asymmetric DCIM variant that applies the region Mamba to the PET branch only (62.26% IoU) is nearly as good as the full configuration, and naive PET-only Mamba variants outperform some ablated settings.
- Swin-UNet and TransUNet perform far below CNN and modern fusion baselines on PCLT20K.
Limitations
- Dataset is 2D slice-based and split randomly at patient level; only lung tumors from one hospital scanner.
- Reported results are single-run with no variance or statistical tests.
- Dataset labels are for tumor parenchyma only and may be discontinuous.
Experimental setup
Datasets: PCLT20K STS
Benchmarks: PCLT20K-TE STS
Metrics: IoU F1 accuracy HD95 FLOPs parameters
Baselines: TokenFusion CEN CMX CMNeXt AsymFormer DFormer Sigma GeminiFusion nnU-Net Swin-UNet TransUNet nnSAM
Builds on: VMamba Mamba Sigma U-Mamba
Closest papers by meaning
- NaMa: Neighbor-Aware Multi-Modal Adaptive Learning for Prostate Tumor Segmentation on Anisotropic MR Images (AAAI 2024 · similarity 0.85)
- Toward a Vision-Language Foundation Model for Medical Data: Multimodal Dataset and Benchmarks for Vietnamese PET/CT Report Generation (NeurIPS 2025 · similarity 0.85)
- Multi-modal Vision Pre-training for Medical Image Analysis (CVPR 2025 · similarity 0.84)
- TumorChain: Interleaved Multimodal Chain-of-Thought Reasoning for Traceable Clinical Tumor Analysis (ICLR 2026 · similarity 0.84)
- LungNoduleAgent: A Collaborative Multi-Agent System for Precision Diagnosis of Lung Nodules (AAAI 2026 · similarity 0.83)
- Training Like a Medical Resident: Context-Prior Learning Toward Universal Medical Image Segmentation (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.