Research map / Vision-language and multimodal / Cross-modal learning
BSAFusion: A Bidirectional Stepwise Feature Alignment Network for Unaligned Medical Image Fusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Hmm
Problem
Fusing unaligned multimodal medical images (CT/PET/SPECT with MRI) requires registration, but registration and fusion have conflicting feature requirements, so two-stage pipelines are complex and single-stage joint methods handle only rigid misalignment.
Approach
Appends a Modality Feature Representation Head to each input and injects each image's head into the other's features to reduce modality discrepancy, then predicts the deformation field bidirectionally in five stepwise layers (forward and reverse meeting at an intermediate point) based on path independence of vector displacement, and fuses aligned features with a multi-modal fusion block.
Key results
- On MRI-PET, the proposed method reports Q_AB/F 0.7770 and Q_CV 67.7 versus the best baseline UMF-CMGR (0.6376, 206.2).
- On MRI-CT, Q_AB/F 0.3927 and Q_CV 4155.1, outperforming IMF (0.2665, 4439.6) and UMF-CMGR (0.2501, 4638.7).
- Ablation shows removing BSFA drops MRI-CT Q_AB/F from 0.3927 to 0.3279 and bidirectional alignment beats FRL-only or RRL-only variants.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BSAFusion instead of the strongest reported baseline | Unaligned multimodal medical image fusion · MRI-CT | Q_AB/F: 0.3298 -> 0.3927 | Helped |
| Use BSAFusion instead of the strongest reported baseline | Unaligned multimodal medical image fusion · MRI-PET | Q_CV: 206.2 -> 67.7 | Helped |
| Use BSAFusion instead of the strongest reported baseline | Unaligned multimodal medical image fusion · MRI-SPECT | Q_AB/F: 0.6463 -> 0.7679 | Helped |
| Include MDF-FR fully rather than Setting A (no MFRH swapping or L_ce2) (ablation) | BSAFusion · MRI-PET | Q_CV: 99.2 -> 67.7 | Helped |
| Include MDF-FR fully rather than Setting B (no MFRH swapping, with L_ce2) (ablation) | BSAFusion · MRI-PET | Q_VIF: 0.3941 -> 0.4189 | Helped |
| Use both forward and reverse registration layers (BSFA) rather than no BSFA (ablation) | BSAFusion · MRI-PET | Q_AB/F: 0.7236 -> 0.7770 | Helped |
| Add the reverse registration layer (RRL) (ablation) | BSAFusion with forward registration layer · MRI-PET | Q_AB/F: 0.6817 -> 0.7770 | Helped |
| Add the forward registration layer (FRL) (ablation) | BSAFusion with reverse registration layer · MRI-PET | Q_AB/F: 0.7272 -> 0.7770 | Helped |
| Use MMFF instead of direct feature concatenation (ablation) | BSAFusion · Fusion benchmark in MMF Performance Evaluation | Q_AB/F: 0.6765 -> 0.7218 | Helped |
| Set the numbers of FRL&RRL blocks and FusionBLK modules to 5 (ablation) | BSAFusion · Fusion benchmark in MMF Performance Evaluation | Q_AB/F: 0.6849 -> 0.7218 | Helped |
Limitations
- Model is not sufficiently lightweight due to multiple FRL/RRL and FusionBLK modules and high parameter count.
- Evaluated only on Harvard CT/PET/SPECT-MRI data with synthetic deformations applied to non-MRI images.
- Ablations show best results only with five FRL/RRL and five FusionBLK blocks, indicating sensitivity to architecture depth.
Experimental setup
Datasets: Harvard medical image dataset (CT-MRI, PET-MRI, SPECT-MRI)
Benchmarks: Harvard CT-MRI Harvard PET-MRI Harvard SPECT-MRI
Metrics: Q_AB/F Q_CV Q_VIF Q_S Q_SSIM
Baselines: UMF-CMGR SuperFusion MURF IMF PAMRFuse U2Fusion EMMA DDFM MATR ALMFnet MsgFusion MDHU
Builds on: Restormer IMF UMF-CMGR PAMRFuse
Closest papers by meaning
- Modality-Agnostic Structural Image Representation Learning for Deformable Multi-Modality Medical Image Registration (CVPR 2024 · similarity 0.82)
- A Robust Mutual-Reinforcing Framework for 3D Multi-Modal Medical Image Fusion Based on Visual-Semantic Consistency (AAAI 2024 · similarity 0.82)
- SSLFusion: Scale and Space Aligned Latent Fusion Model for Multimodal 3D Object Detection (AAAI 2025 · similarity 0.82)
- Adaptive Bidirectional Displacement for Semi-Supervised Medical Image Segmentation (CVPR 2024 · similarity 0.82)
- UniMRSeg: Unified Modality-Relax Segmentation via Hierarchical Self-Supervised Compensation (NeurIPS 2025 · similarity 0.82)
- Multi-modal Vision Pre-training for Medical Image Analysis (CVPR 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.