Research map / Architectures and efficiency / Attention and transformers
DDViT: Double-Level Fusion Domain Adapter Vision Transformer (Student Abstract)
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DDViT, a hybrid CNN–ViT segmentation model with a plug-in domain adapter and double-level fusion, evaluated on EBHI-Seg colorectal histopathology images.
Problem
ViTs require large datasets and capture low-level features poorly, limiting medical image segmentation performance on small datasets.
Approach
Combines a CNN encoder with a TransFuse-style dual-branch transformer, adds a domain adapter that uses the multi-head self-attention to inject domain-label information, and fuses multi-scale features via a double-level fusion (DLF) module with a semantic-FPN-style decoder; mutual knowledge distillation between universal and domain-specific branches is mentioned in the abstract.
Key results
- On EBHI-Seg, DDViT reaches Dice 0.725 on Normal vs 0.777 for Seg-Net, 0.411 for U-Net, and 0.554 for TransFuse.
- DDViT achieves the best Dice on serrated adenoma (0.951) and low-grade IN (0.937) among the four compared methods in Table 1.
- DDViT is below Seg-Net and U-Net on Polyp (0.904 vs 0.937 and 0.965) and on the Normal class is below Seg-Net (0.725 vs 0.777).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DDViT instead of Seg-Net | EBHI-Seg segmentation · EBHI-Seg — Normal | Dice ratio: 0.777 -> 0.725 | Hurt |
| Use DDViT instead of U-Net | EBHI-Seg segmentation · EBHI-Seg — Polyp | Dice ratio: 0.965 -> 0.904 | Hurt |
| Use DDViT instead of U-Net | EBHI-Seg segmentation · EBHI-Seg — High-grade IN | Dice ratio: 0.895 -> 0.902 | Helped |
| Use DDViT instead of Seg-Net | EBHI-Seg segmentation · EBHI-Seg — Low-grade IN | Dice ratio: 0.924 -> 0.937 | Helped |
| Use DDViT instead of U-Net | EBHI-Seg segmentation · EBHI-Seg — Adenocarcinoma | Dice ratio: 0.887 -> 0.901 | Helped |
| Use DDViT instead of U-Net | EBHI-Seg segmentation · EBHI-Seg — Serrated adenoma | Dice ratio: 0.938 -> 0.951 | Helped |
Surprising findings
- DDViT underperforms Seg-Net and U-Net on the Polyp class and Seg-Net on the Normal class in Table 1, despite overall claims of superiority.
Limitations
- Evaluation is limited to a single dataset (EBHI-Seg) in a short student abstract.
- Improvements over baselines are described as modest, and DDViT trails some baselines on certain classes.
- Ablation studies of the domain adapter, DLF, and distillation are not reported.
Experimental setup
Datasets: EBHI-Seg
Benchmarks: EBHI-Seg
Metrics: Dice ratio
Baselines: Seg-Net U-Net TransFuse
Builds on: TransFuse MDViT domain adapter Semantic FPN
Closest papers by meaning
- Your ViT is Secretly an Image Segmentation Model (CVPR 2025 · similarity 0.87)
- DTMFormer: Dynamic Token Merging for Boosting Transformer-Based Medical Image Segmentation (AAAI 2024 · similarity 0.87)
- nnWNet: Rethinking the Use of Transformers in Biomedical Image Segmentation and Calling for a Unified Evaluation Benchmark (CVPR 2025 · similarity 0.87)
- E2ENet: Dynamic Sparse Feature Fusion for Accurate and Efficient 3D Medical Image Segmentation (NeurIPS 2024 · similarity 0.86)
- MedSegDiff-V2: Diffusion-Based Medical Image Segmentation with Transformer (AAAI 2024 · similarity 0.86)
- Channel Vision Transformers: An Image Is Worth 1 x 16 x 16 Words (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.